This blog post aims to share the discussions and reflections of the webinar presented by Dr. Shannon Johnston (refer to Webinar about Evaluating Design-Your-Own-Module (DYOM)). In her discussion on how to evaluate a DYOM, she shares her experience of redesigning a third year Indonesian language curriculum as a case study.

Framework

The following considerations can be used to guide evaluation of DYOM. 

Firstly, one should identify the Purpose(s) of the module or curriculum. Examples of a purpose could look like ‘to prepare graduates for futures with the language’ and ‘to be interesting, different, and to motivate’. 

Secondly, Data has to be collected. Data could come in many forms: from observation of student behaviour to collected feedback. 

Thirdly, Evaluation, judged based on whether the Purpose(s) are achieved, and what the Data suggests.

On Analysing Data

Questions when faced with collected data are as follows:

1. What might the data tell us?
2. Whose story does the data tell? (i.e., who is included and who is left out)

Taking a step back, it is also important to examine not only the data, but what data was collected, why that data, whose voice is privileged by the data, and what other data should be collected.

On Collecting Evaluative Feedback

There is not one way to collect feedback and nothing dictates that feedback can only be collated through one method. For example, students can take part in a reflection session in which they discuss the role they play or what they learnt. Leading questions can be prepared.

Other ways of collecting feedback include:

• student daily blogs
• student final assessment item
• student focus group interviews

On Design Thinking Methodology

The Design Thinking methodology can be used in evaluating Design-Your-Own-Module. For example, referring to Stanford’s Design Thinking, the following 5 main steps of (1) Empathise, (2) Define, (3) Ideate, (4) Prototype, and (5) Test can be used. The progress may not be linear and one may find oneself going back to the drawing board or a few steps back in order to come up with a better design. For instance, the prototype may not work in a way that one expected or the ideation process would have to be used again in order to find a better solution.

Design Thinking also emphasises 3 fundamental values – empathise, observe and identify. With reference to the field of teaching and learning, staff should be student-centred. Questions like ‘what was it like for the student?’ and ‘what was the students’ experience?’ are crucial.

Identifying latent needs is another important lesson from Design Thinking. This links to a previously discussed goal of DYOM, which is to tap on the intrinsic motivation of students (refer to Webinar about DYOM via Group Work with Supervision). This gets staff to probe deeper into students’ core motivators outside of ‘completing the module’ or ‘getting a good grade for the module’.

Parting Questions for Evaluating DYOM

• What worked well?
• What did not work so well?
• What did we not do (facilitators or educators) that we should have done?
• What will we do now?

Presented by Dr. Shannon Johnston,

Head of Professional Learning, Murdoch University

The webinar is organised by the NUS DYOM Learning Community.

Slides

Loading...

Taking too long?

Reload document

| Open in new tab

Download

The recording of the Webinar is uploaded in parts of less than 50 MB each due to upload constraint.

Webinar Recording 1

Webinar Recording 2

Webinar Recording 3

Webinar Recording 4

This blog post aims to share the discussions and reflections of the webinar presented by Professor Alison Cook-Sather (refer to Webinar about Student-Faculty Pedagogical Partnership in Module Development).

Benefits of Pedagogical Partnership

Pedagogical learning, as argued by many researchers and educators, transforms the campus into a place of belonging for the students (Colon Garcia, 2017). It also encourages values such as responsibility, respect, trust, enhancing satisfaction and personal development (Lubicz-Nawrocka, 2018). Such a learning style could promote more effective learning, not just for the students but the educators.

Real life experiments in various universities demonstrates that collaboration with students also received praise for increasing inclusivity. Students can voice out concerns and challenges and feel that their voice matters in their educational journey. Students do not feel that higher education as a process of receiving work and submitting it.

Students remark that they felt valued and can practice at working democratically. Curriculum becomes more socially relevant. This form of teaching enhances students’ identity and metacognitive awareness of learning and teaching.

Other benefits of partnership are as follows:

• Feeling more engaging and relevant
• More equitable and inclusive content and practices
• Increased commitment to transparency in curriculum design
• Greater confidence and willingness to take risks
• Recognition of all students as partners

Ways to Conduct Partnership

Different educators have different circumstances and classroom objectives. Hence, there is no one correct way to engage students as partners. Three methods were shared and discussed.

Firstly, the one-on-one partnership method. This method involves picking one student and working with them throughout the semester to gather feedback regarding the module and having him/her be involved in the curriculum processes, such as having the student share his/her thoughts on possible learning outcomes, learning activities, assessments, and due dates. However, there are concerns that this method may be biased and not reflective of every student’s needs. Student concerns are not monolithic, and this method could blindside the educator.

Secondly, the whole-of-class co-creation method. This method involves the entire class be invited to collaborate. This method is more inclusive as it gathers the feedback from everyone involved. Despite such benefit, the concern is that all students may not be eager to participate.

Thirdly, an opt-in basis for the students. Rather than singling out a student or forcing everyone to be involved, the educator could open the floor up to willing participants.

Parting Thoughts

• What outcomes for your faculty are you interested in and why?
• What outcomes for students are you interested in and why?
• Which course could benefit from a one-on-one, whole-of-class, and opt-in method?

References

Colón García, A. (2017). Building a Sense of Belonging through Pedagogical Partnership. Teaching and Learning Together in Higher Education, 22, https://repository.brynmawr.edu/tlthe/vol1/iss22/2.

Lubicz-Nawrocka, T. (2018). Students as partners in learning and teaching: The benefits of co-creation of the curriculum. International Journal for Students as Partners, 2, 47-63, 10.15173/ijsap.v2i1.3207.

Presented by Professor Alison Cook-Sather,

Mary Katherine Woodworth Professor of Education, Bryn Mawr College

Director, Teaching and Learning Institute, Bryn Mawr and Haverford Colleges, Pennsylvania

The webinar is organised by the NUS DYOM Learning Community, with support from NUS Educator Development Fund.

Slides

Loading...

Taking too long?

Reload document

| Open in new tab

Download

The recording of the Webinar is uploaded in parts of less than 50 MB each due to upload constraint.

Webinar Recording Part 1

Webinar Recording Part 2

Webinar Recording Part 3

Webinar Recording Part 4

Discussed at the midpoint of the Learning Community. This blog post is prepared by members of DYOM LC (find more in About Us).

This blog post aims to share the discussions from the webinar delivered by Professor Peter Felten on Student-as-Partners (SaP) and the reflection of the Learning Community with regard to the topics that have been discussed so far. SaP framework was particularly explored following past discussions that pointed to the importance of a strong partnership between students and staff for a successful DYOM.

Considering Students as Partners and Applying SaP

Students having agency over their learning is important and enables students to develop their ability to think critically. The reciprocal benefits of partnership between students and staff are as follows:

1. Engagement, i.e., enhancing motivation and learning
2. Awareness, i.e., developing metacognition and identity 
3. Enhancement, i.e., improving teaching and experiences 
4. Belonging, i.e., cultivating meaningful communities  

Undergraduate students take modules that are mostly decided by the faculty. Students and staff are mostly not accustomed to partnering and may find partnership out of their comfort zone. To conduct a meaningful DYOM, students and staff may have to unlearn some habits about learning. As actors and agents of their own learning, students may make mistakes. This is part of the learning process. The desired outcome is not a perfect DYOM, but that the students have achieved the learning objectives and acquired the confidence to be active agents in their learning process. 

SaP in Practice

Practicing SaP includes having meetings in students’ study spaces rather than staff’ spaces, as physical environments also carry the message that DYOM is where decisions and power are shared between students and staff, unlike other modules. 

Enhancing interactions with students includes checking in with students regularly (the wisdom is every two weeks) about what is and is not working and getting students to make recommendations.

In designing the modules, applying the Backward Design (refer to this post) for module preparation will be useful, as follows:

1. Articulate the learning outcomes
2. Identify what achieving these outcomes looks like (i.e., assessments)
3. Plan learning experiences to achieve the intended learning outcomes

Points for Further Discussions

In the context of DYOM in NUS, staff are still the ones conducting assessment and grading at the end of the module. At the start of DYOM, staff are the ones who will need to agree that a DYOM proposal has a sufficient scope. Moving forward, a discussion point is developing a practice such that the design of the module can be done by students as much as possible. 

Another discussion point is about balancing between allowing students to have ownership of their learning and ensuring that students learn what they need to learn. Although staff seem to have taken a backseat in the learning process, staff still need to bring to students attention lessons that are good for students to learn. In the context of DYOM, the role of staff is that of being present and guiding figures in students’ learning. The discussion point touches on the ethics of teaching and learning. 

Conclusions and Follow Up 

LC members would like to apply SaP to their respective DYOM. There may be some difficulty in the initial stages of implementation due to unfamiliarity with SaP and comfort with the typical way of module design. “Unlearning” is a powerful message to start the journey towards effective partnership between students and staff.

Discussed during and after the presentation of Mr. Seah Zong Long and Mr. Pok Ruey Jye. This blog post is prepared by members of DYOM LC (find more in About Us).

This blog post presents the experience of facilitators and co-facilitators. Members of DYOM LC discussed methods that work and challenges that need solving. Members also discussed the different roles of module coordinators and facilitators and mechanisms that one could adopt to create a sense of ownership in a DYOM. 

The general comments were that the presenter’s DYOM succeeded in conducting interdisciplinary learning. Students learnt technical and academic aspects, and this was made easier by the fact that the DYOM had facilitators who had their expertise in the different aspects. When conducting the DYOM, the facilitator and the module coordinator synthesised their knowledge about different areas of the topic and found that this helped enhance students’ learning. This sharing prompted LC to think about:

1. What does it mean to facilitate DYOM?
2. What does it mean to teach DYOM? 

The importance of making the DYOM journey interest-based and experiential was also emphasised. For instance, field trips to a vintage camera shop were found to be useful for a photography DYOM in providing students with knowledge about the equipment and meeting the photography community. 

The presenter acknowledged that there are strong cultures at the backdrop of the respective DYOMs – for instance, the photography or business DYOMs received students who already had prior interest before taking DYOM. 

Effective Methods in Running DYOM

Each DYOM has varying degrees of ownership within its students. After rounds of discussion, the LC recognised the following methods as effective ways in making the students comfortable in taking charge of their own learning:

1. Peer-graded sessions
2. Students being involved in creating rubrics 
3. Seminars 
4. Conducting a pre-survey on why students take the particular DYOM 
5. Obtaining feedback on the experience interacting with professionals 
6. Obtaining feedback about DYOM after completion
7. Integrate Student-as-Partners Framework (more of this in Blogpost #3)

Lessons from Student Experiences 

Upon reviewing student feedback for various DYOM, the LC noted a similarity in student feedback over multiple DYOM – that many would comment about the unexpected workload. Students expressed their desire for the workload to accurately match the Module Credits. A 2 MC module should therefore have 2 MC worth of work in the module, as students often plan their other modules according to this guide. 

Another challenge was the diversity of students. For example, having students from different faculties or majors means having students who have differing prior knowledge. An entrepreneurship module, for instance, used many concepts and technical terms whom students from Business were well-versed in, but not students from other faculties. There was hence a stark difference in what the students experienced – some found the material too simple; others, too complicated.

Student feedback also indicated that some students preferred having the material curated for them, while other students felt that they were not involved enough in the gathering of material for classes. 

Conclusion and Follow-Up 

The discussion has shown that meeting the needs of various groups is one of the biggest challenges in DYOM, be it between students, staff and external speakers, or just within different groups of students. 

The LC also concluded that the way the DYOM is conducted is never set in stone. It is evolving even during the semester, not only after each DYOM has been completed. Improvements can be made as the DYOM progresses. Periodic surveys play a part in helping students and staff communicate and work together more effectively.

Presented by Professor Peter Felten,

Assistant Provost for Teaching and Learning, Elon University,

Director of the Center for Engaged Learning, Elon University, and

Professor of History, Elon University

The webinar is organised by the NUS DYOM Learning Community.

Slides

Loading...

Taking too long?

Reload document

| Open in new tab

Download

The recording of the Webinar is uploaded in parts of less than 50 MB each due to upload constraint.

Webinar Recording Part 1

Webinar Recording Part 2

Webinar Recording Part 3

Webinar Recording Part 4

Presented by Dr. Andi Sudjana Putra, Mr. Alan Soong and Ms. Tham Chuey Peng.

This presentation shares about Design-Your-Own-Module (DYOM) via Group Work with Supervision. The presentation outlines the various topics that have been offered as DYOM, principles that may be used to underpin DYOM and how to operationalise it. The webinar is organised by the NUS DYOM Learning Community.

Slides

Loading...

Taking too long?

Reload document

| Open in new tab

Download

Discussed during and after the presentation of Dr. Sew Jyh Wee. This blog post is prepared by members of DYOM LC (find more in About Us).

This blog post presents the interests and concerns of students in DYOM discussed by the Learning Community (LC). Apart from discussing what students want, this blog post discusses the manner to gather what students want and building rapport with students.

Obtaining Feedback from Students

A clear communication between staff and students is identified as a critical step for a good learning experience. This is especially relevant in DYOM where there is a higher partnership between staff and students. Obtaining useful feedback from students is of utmost importance. The LC discussed how this may be done effectively. 

Student feedback was mainly about student’s reflection on their learning experience and perceived workload. Requesting students to fill in short feedback forms may not capture students’ views towards the module accurately. On the other hand, personal blog reviews that some students publish may be a better reflection of their experiences, in that they are effective, constructive and thorough.

On the topic of gathering feedback so that it can be used effectively, the LC pondered ‘How can we allow students to understand the importance of giving feedback and reflecting?’. This question echoed the earlier point about getting useful feedback rather than feedback that is written for the sake of completing a feedback form. The timing of feedback was to be considered, too; whether at the start, middle or the end of the module. Peer assessment was also considered.

Surveys can be sent out to the students, provided they are modified to suit the topics or concerns at hand. There is interest in developing feedback forms that are suited for DYOM.

Students’ Consideration When Deciding to Read Which Modules, Including DYOM  

The academic year of a student affects the way a student chooses which modules to read. Students consider factors such as workload, interest, how a module helps fulfil graduation requirements (core or elective) and the effect a module has on their CAP (Cumulative Average Point), among other factors. Students also wish to balance the workload of modules taken in the semester, i.e. some ‘difficult’ modules taken together with ‘not difficult’ modules. These considerations affect students’ decision whether to read DYOM. 

On the other hand, given the nature of DYOM, students’ interest in the module is the main motivating factor, although workload cannot be ruled out as one of students’ considerations. 

Rapport with Students

The LC also learned that although tools such as surveys and blog posts are important in gathering students’ feedback and building partnership, a good rapport with the students is an important ingredient to a good partnership. In all the sharing that the LC has heard, a good rapport with the students has always been helpful in motivating the students in DYOM and in enhancing engagement. 

In the beginning of DYOM, modules were started by a few students who took the lead and had a good rapport with the lecturer. Other students followed. This strengthened the notion of relationship and having significant informal networks. As a module progresses, partnership between students and staff will then grow.

Conclusion and Follow Up 

Sharing of student feedback has revealed insights about what considerations matter to students. In order to capture more comprehensive views of students, a more thorough format of feedback will need to be designed.  

In addition, it seems that students choose modules based on extrinsic motivation such as grades and workload. As DYOM is for students to explore their interests, tapping on students’ intrinsic motivation is to be considered.

Discussed during and after the presentation of Dr. Andi Sudjana Putra, Mr. Elvyn Sim and Mr. Muhammad Haizuruldin. This blog post is prepared by members of DYOM LC (find more in About Us).

This blog post presents the Students-as-Partners framework used to study student engagement in DYOM as discussed by the Learning Community (LC).

Theoretical Underpinning

Students-as-Partners (SAP) is used as the framework in studying student engagement in DYOM. The resources that the LC uses are as follows:

• From Griffith University: access from here.
• From Elon University: access from here.
• From The Higher Education Academy (THEA): access from here.

Student engagement can be seen from the following aspects:

1. Students are engaged when they invest time and energy in their own learning. The measure of student engagement can therefore be based on the time and energy they invest in learning.
2. Students are engaged when they are involved and empowered by the institutions (e.g., their schools) to shape their learning experiences.

According to the THEA (refer to Figure 1.1 of the reference), the four stages of student engagement are as follows: 

1. Consultation
2. Involvement 
3. Participation 
4. Partnership 

Upon reviewing these theoretical concepts, the LC related to their own DYOM experiences. For example, some experiences point to consultation taking place in the form of informal conversations outside the classroom and participation in the form of designing assessment rubrics with students. 

Observation and Operationalisation

Notable points were raised according to the experience of LC members as facilitators, administrators and students. 

Firstly, there has been a desire to enhance an experiential component in DYOM, hence making DYOM distinct from other modules in NUS. Experiential components include students designing (or co-designing) a set of assessment rubrics and syllabus with their professors. This shows that there was indeed a higher level of student engagement in DYOM than in other modules as students have more say in their own learning. Having observed this, it was also shared that there have been instances where students accepted a proposed DYOM plan from their professors without much objection. This might be because students were passive, felt lacking in knowledge about the topic to make constructive contributions, agreed with the proposal or simply being grateful for the module. 

Secondly, student engagement may not necessarily be at the Partnership level, or that it may not always be aimed at Partnership level. Contextualisation is required to determine the proper level of student engagement for various activities in a DYOM. For example, student engagement in deciding the mode of learning may not be the same as in deciding assessment modes, in turn may not be the same as when creating content and so on. 

Thirdly, providing appropriate pedagogical consultancy to students is both an opportunity and a challenge.

Although every DYOM topic is unique and participants of each DYOM would have had different experiences, there were some similarities and useful observations for the LC. 

DYOM via Group Work with Supervision can be viewed as ‘the marrying of MOOCs (Massive Open Online Course) with experiential learning’. It became evident that the element of experimenting or customisation strongly defined the DYOM experience. 

In terms of the level of student engagement, the LC concluded that each DYOM has elements of all four levels, although to varying degrees. It is worthwhile to think about increasing the level of student engagement. Each DYOM may need to consider its own context. For example, a DYOM organised by Office of Student Affairs (OSA) that requires students to run a project in their halls of residence will need to engage all stakeholders: not only students and OSA staff as the organiser, but also hall staff. 

Conclusion and Follow Up

There is no ‘one size fits all’ that would work for every DYOM or for all groups of students. Contextualisation is therefore important – to tweak and adjust the curriculum to meet the needs of the stakeholders for the best experience. 

The LC also reflected on the underlying motivations of the students; i.e., on the importance of describing to the students that DYOM is to be seen as an avenue to pursue their academic interests, rather than only as a ‘chill’ module for them to fulfil Unrestricted Elective (UE) requirements. Students’ learning experience would be more pleasant once they start steering their module in this direction, seeing that the DYOM aims to do both – cater to their interests while being enjoyable.