Learning Through Feedback

Happiness is one of those topics that would spark curiosity and be of interest to anybody of any background. To me, happiness is one of those things that is difficult to measure; these are things you can explore in much more depth. If you were looking at something that is just a deterministic process, it wouldn’t lend itself to be explored in different ways and across different backgrounds.

As an urban and transportation geographer, I was familiar with the notion that active modes of transport create more happiness. People who can walk and cycle to work tend to be happier than people that are stuck in traffic. But that was just a little kernel of happiness that I knew, so I had to learn more about it myself. I did a bunch of Coursera courses and quite a bit of reading as well. With that, I came up with questions that you could pose to anyone on a more generalised level. That led me to the World Happiness Report, which gave me a barometer to show how certain countries are happier than others, and whether or not countries were becoming happier in general.

I built the class from there because once you have that dataset, things can start to emerge from exploring that dataset, from the narrow perspective of a transportation geographer to something broader like the World Happiness Report, covering every country in the world over 20 years or so with several variables. Once we dig into this dataset in class, the rest of the exploration and quantitative reasoning is something that people can build up on their own. To me, if you can understand one dataset, you can easily make that jump to understand others.

It has changed quite a bit. When I first started teaching, it was very structured: there was a rubric, a mid-term, a final, a presentation, and a paper. It went fairly well, but it was still my first semester so it wasn’t perfect.

One of my students came to me and told me “When I came to USP I thought that we would be doing something different, but it’s like the same lectures that I got in the rest of NUS”. I thought that USP classes should have a legitimate difference from the other classes in NUS, where people can say, “Well, I like this USP class” or “I don’t like this class”, but both statements can agree that USP classes are different. 

So from that students’ input and seeing USP students in a very well-structured class just acing the module in terms of grades, I thought it wasn’t a fair way to measure students of high aptitude and high effort, because everyone was scoring “A”s and “A+”s. It was hard to find the gaps between high-calibre students through these standard, highly structured metrics. Because I saw that students were doing great with structured modules, I wanted to value-add by giving them an unstructured module. I wanted to expose them to a different learning environment, just in case they ever end up in an unstructured environment like a job or internship where bosses just give you a stack of paper or some broad overarching goal, and you have to figure things out yourself. 

At first, when I took a lot of the structure out, it was too much, so I found some structure to put back in. But instead of thirteen weeks of “here’s what we’re going to do, this is how your assignments are going to be scored”, the module was built around data and software. There are the core learning objectives I identified: learning how to work with data, learning how to ask questions and learning how to add context to your results. It was aiming for the fine balance of having enough structure where there were things that we were doing together, but not enough structure to the point where students are going to score 90% because they’re so good at understanding instructions.

I think it varies depending on what type of industry and office environment that you’re working in. I’m assuming that if you’re in a bank or civil service, there are a lot standard operation procedures. But if you go into something like sales, it would come in handy. 

For DMI [Developing Meaningful Indicators], it’s almost like sales. You have a market, like a subreddit or Reddit, that will give you feedback based on whatever product you put into the market. Because of that, you have to figure out what the topic is at that time, what’s the data you are going to use, what the visualisation type is, what colours you have to use. There’s just so much that you have to figure out on your own. DMI is relevant if you become a salesperson or an entrepreneur because it is at the absolute far end, where the objective is to get students to be able to go into the marketplace, to get people to “purchase” it, to get people to discuss your visualisation and the topic.

It’s also about getting feedback on your product, from me particularly, but also from other people. I think that’s the most overlooked thing that you can get out of USP. That’s probably the best value from small classes: rather than input from professors, it’s the feedback on what you output that’s way more important. In a normal module, you have limited opportunities to get feedback, perhaps only twice in a semester. In reality, you should have to get feedback weekly, daily even. It might be an essay, a report, or a data visualisation. Things are much more iterative now and that feedback component is really important. For me, the lack of structure in things should be made up by the feedback that you get. 

For DMI, I typically give between a thousand to two thousand messages of feedback each week, but I think this is undervalued by students and higher education. I don’t really understand the value of higher education as gatekeepers or structures of knowledge. When it comes to things that are complex, I think that feedback is really necessary. Presenting things to broad audiences, not only to your teachers but to people you don’t even know, these things are really complicated and you only get better at those things through receiving feedback. 

But even when I’m providing feedback, I can’t guarantee the students that if they check certain boxes, then people will definitely respond well to that. Rather, it’s about testing how it appears to an audience and seeing if that visualisation is truly meaningful and insightful to an audience. To make these kinds of things very structured, When it comes to making a data visualisation, you need people to discuss it. I’ve seen the ugliest bar charts generate a lot of discussion and the most beautiful and interactive posts get very little discussion. It’s untangling why that is. Is it the timing? The topic? That’s what the feedback is for. Sadly, this opportunity to get feedback directly is undervalued in higher education.

There is a lack of opportunity to give feedback, like for WCT professors, they give very in-depth feedback, I know they put in a lot of effort in correcting your draft and give one-on-one consults, that’s really useful. But to me, there should be more opportunities to connect with your professors and get feedback. To me, introducing Slack was for me a way to connect with students and guide them at every point of their process. 

Unconsciously, I know that some students struggle when the focus is on feedback, student output, and their own drive rather than driven by the lectures, deadline and the module itself. This unconventional flip of you having to do things first and I come in to help you, I try to direct that discomfort more towards me. I know there is this misalignment and it’s always a question of “am I comfortable enough with that misalignment?” but I do feel bad sometimes. As a lecturer and professor you don’t want your students to feel uncomfortable. But I do think that it’s helpful for some students in the long term. That is the kind of impact that you won’t be able to get in a lot of the other classes.

So the question becomes “Am I comfortable enough that this misalignment is at least going to help some students?” From a subconscious perspective, I understand that it is not something that is for all students. I understand that not all students want this approach of getting feedback. But when I look at my colleagues, and what other educators are doing, I don’t think every class will benefit everybody, and some classes don’t even benefit anybody. There are some classes that you just do well in, but it doesn’t really add any value to you in the long term. Basically, what I’m saying is that the bar is very low to take an approach that doesn’t work for everybody. I think in some cases you are going to take classes that works for everybody, but you’re not going to take away a lot from it.

The bottom line is, different classes have different levels of benefits for students. You have classes where you actually don’t take away anything. Enjoyment does not always mean you’re learning. The question for me then becomes: “Are you willing to take a risk that might not work out?” Because the alternative to me doesn’t seem more palatable:a more structured class that students might like but people might not take anything from it. This really ties back to when the student approached me and said that your class is great, but I thought that it would be different. And so I strive more to be different than to be anything else, really. 

While I seem to be dividing students into two groups, as though some are better than the others, in reality, I feel that it works best for every single USP student. But the real dividing factor is definitely time. My modules really don’t work for students who have an overloaded semester, or taking on an internship. If I’m putting everything onto you for this module, then it is that extra buy-in that you have to do, that you might not have the time for. Some students are in Medicine or Law, or taking a lot of classes, or taking difficult courses. In order to get feedback and do things iteratively, you need to participate quite a bit. Understand that even though there is a lot of self-learning, this can be guided even if you plateau, or when you go in the wrong direction. You have to take on that journey yourself. It’s those who want to learn and have the time and capacity to invest in the course, it’s those who have the don’t-care attitude of “I’m going to participate in this class even though it might not be that significant to my grade”. As long as you invest yourself to this course, I think you’re going to get something out of it. I think that that’s what higher education should be in general, but at the same time I know that modern education wants to teach students as much things in as little time as possible, and balance it with other “competitive workforce” stuff like interest groups and internships. I found that that’s the biggest separator, that many students are really overloaded.

Also, the willingness to take risk and make mistakes is also very important. I want to encourage this, but I think that in every young person there’s a bit of unwillingness to do things your own. The willing student is the one who’s willing to take steps on their own and make mistakes. All these things that I said, I don’t do things because I feel it’s the right thing to do. I try to take cues from educational research and also from the goals of USP – this idea of being independent, adaptable thinkers and doers, who would make an impact on the world. When I look at those things, I ask myself, how do I measure the ability of students to be independent? I have to give you a certain degree of independence to do that. And how can I see that you are willing to adapt? Again, these are all individualistic things that have no standardised measure.

There’s this idea of making an impact on the world: you’re presenting your work not to your class. That’s why I bring in Reddit. If one person comments on some bar chart that you make, to me that’s some impact, as compared to me giving you an “A” on the bar chart you present, where your impact is limited to just the classroom. What is the smallest impact we can make, what is the largest degree of latitude for independence, and how can I ensure that you can adapt to different environments in work? If everybody went for those core programme objectives, they would shape their module like this too.

But there are some trade-offs: if you’re looking for independent, adaptable doers, it will change the whole structure of higher education and people aren’t going to like it. You’re going to have to accept that. People’s jobs are at stake; student feedback matters on teachers on the teaching track. There’s no blame on anybody, because you have this huge misalignment between what everybody wants from higher education. If you can get all of these things to align – students, professors and higher education – then it can really work. If any one of those things misalign, then it’s not going to work at all. You have to tolerate that failure.

Very difficult. I had to read a lot in terms of emergent structures and things like that. That was the course I feel suffered more. The DMI course is what it is, you make your products and release them to the world. That’s just the nature of communication. For QR, there are things that you need to know. To figure out what those things are without making those determinations myself in a standardised way, but at the same time allowing your own individual reasoning to emerge from some structure. I had to figure out what those things were.

I really sat down and thought: data, software, analysis, and context became the four things for that module. I’m still working to encourage that emergence among those four things and it might not be enough, or it might be too much. Getting to those four constraints was difficult. I’ve tried both extremes of 0 constraints in the class and 13 constraints (one for each week) and neither of them worked. Right now it’s 4, but I’m still not confident that it’s enough, maybe there should be 5 constraints. I’m still trying to figure out what these specific things are, to allow students to have their own emergence from those things. When you do the same things, then everyone’s takeaway will be the same. Knowing what these constraints are takes a lot of time.

I think you start to see those strengths and weaknesses emerge when you remove some of those specific constraints: you can identify where the weaknesses are and encourage those who are much better. English majors are very good at communicating their findings in a written format. Social science students are really great at adding context. They can start with some theory about happiness, say cultural influences, and build off from that. The more mathematical students have trouble with that: they struggle because they have a structured way of how they work with data and connecting data with their findings, so they can’t deviate easily. Connecting with theories and writing their findings is not something that comes as easily to them.

To mix these strengths and weaknesses together, I think it’s really useful. That’s why we use Slack to record these findings on a portal that allows everyone to see each other’s minds. I can see the sociologists just spending too much time on theory, so I can guide them: “We’re talking a lot about theory here, let’s go find a data set to see if the theory matches the findings.” To a data scientist who’s not bringing in any context at all, I’ll ask them to think about what connections they make rather than focusing just on the variables. Once you identify these strengths and weaknesses you can guide them towards whatever they are not incorporating and build up their competency.