Hello! I hope your recess ‘recess’ week has been good…
Hope that little bit of humour makes the weekend before midterms a little better. 🙂
So I know I mentioned that I was going to start on talking about how the different religions in Singapore delve into issues related to the environment, but after Dr Coleman gave some feedback on my survey method, I thought it would be good to spend one more post exploring how I could have done it better.
First up – the survey questions:
As you may notice, there were some problems with my questions.
Firstly, I used too wide a scale – from 0 to 10 – which prevented me from being able to allocate a quality to each number, making any number between 1 and 9 subjective, as I cannot place a qualitative value to any of those numbers. This means that someone’s 7 could be the same as another person’s 4, and I wouldn’t know!
Instead, what I could have done is used a Likert Scale, which looks like this:
This would have allowed me to better analyse the data and make stronger conclusions as it is quite clear what each point on the scale represented. It’s a pity that I sent out the survey before I read Dr Coleman’s advice on using said method… but I’m going to use in research for my sustainability project!
Next, some of the questions that I asked were phrased unclearly. An example would be “To what extent do you think is the environment discussed in your religion?” – not only does this assume that the reader knows the full gamut of environmental issues, but it also leaves it open to interpretation on whether discussion refers to the environment being talked about in religious teachings, or whether religious teachers in Singapore talk about the environment.
Two people also raised up that the question “To what extent has not having a religion affected your stand on the environment?” is difficult for areligious respondents to answer because they would first have to understand how religion affects their views. Therefore, even though 81.6% of areligious respondents said that not having a religion had 0 effect on their environmental stance, it may be due to them simply not knowing how it might have affected them. After all, most of the remaining 18.4% who answered between 1 to 10 mentioned that they had a religious childhood or grew up surrounded by religious friends, and thus had a basis for comparison. However, I’m still not sure how to better craft this question. Any thoughts?
Lastly, one issue that I realised upon reflection is that, in doing this survey, I am indirectly discounting those who may not have the language ability to comprehend my survey questions. What led me to think about this was when one of my friends who does not have a good command of English said that she could not understand my questions (even the first one on whether one subscribes to a religion) and had to ask for help to finish it. Even though I attempted to simplify the language I used, it still reflected my assumptions of what is deemed “simple English” – which might not really be that simple.
Perhaps this was another reason for the higher percentage of areligious respondents. (Here’s a study that found every additional year of education reduces the likelihood that an individual identifies with any religious tradition.) But I think the bigger question is:
Did I indirectly contribute to the perception of inequality in environmentalism?
I don’t think that I’ll find a definite answer to that, but it has definitely made me more aware of the importance of nuance and language use in crafting surveys – hopefully, my research in the future will be better done!
Till next week!
-Dennis
(Cover photo: @rawpixels from Pexels)
Hi Dennis,
This analysis is great. I was most intrigued by the paragraph that starts with…
“Two people also raised up that the question “To what extent has not having a religion affected your stand on the environment?””
One thing you could have asked instead (maybe)…
“Are your views on the environment at all linked to the fact that you describe yourself as an atheist?” Maybe with a likert scale, and then a follow-up, open-ended question like “Could you please explain a bit about your response to the previous question?”
I think the question you ask at the end is also very interesting and reflective.
I disagree with your statement that it’s a pity you didn’t show me your questionnaire beforehand. We know from abundant evidence that we learn deeply from our mistakes, i.e., from reflecting on them in hindsight. Srsly, it’s better for you that I didn’t give you too much feedback ahead of time – this way you can look at the responses you got and figure out what worked and what didn’t. Had I helped you craft the questions, you would have just been “taking my word for it”.
Something else… you didn’t have time for this in this assignment, but a very common approach to social science is mixed methods. You start with focus group discussions, where you ask people in small groups to talk about certain topics (you guide them). You gather qualitative data, which you can then code and use to ID key themes. But you can also use the data to craft effective questions for a survey, where you gather quantitative data and can do stratified sampling. This is how several of my students have done their honours research. Here’s a paper you might be interested in if you want to learn more about this (no pressure).
https://link.springer.com/article/10.1007/s13384-016-0216-5
jc
Thank you for the valuable insights Dr Coleman! After reading the article, I find that the usage of focus groups is also a really interesting avenue to explore, seeing that it not only helps craft better surveys, but also gives me more material to work with – will definitely work it into future studies.