The teaching of language and communication involves not only impactful lessons, but also carefully constructed assessments. SoTL Matters recently approached Associate Professor Li-Shih Huang from the Department of Linguistics, University of Victoria, to answer some questions regarding assessment from the perspective of a practitioner. Our questions for her were shaped based on her recent paper about critical dialogues on assessment with practitioners, published by the Journal of English for Academic Purposes.
Prof Huang: I am so delighted to know that the SoTL group at the Centre for English Language Communication, National University of Singapore, has selected my article on assessment literacy, and to learn that the findings of the article resonated with them. My thanks for their dedication to SoTL and for their questions in response to their reading of the article. I hope that the following responses can spark more thinking and dialogue about ways to navigate the impacts of testing and assessment in our practices, where priorities can be easily steered away from, as the study reported, the primary purposes of assessment identified by practitioners, namely, to inform teaching and student learning in order to benefit, first and foremost, our students.
SoTL Matters: What is the role of reflection for language and communication practitioners when they are engaged in developing assessment literacy?
Prof Huang: This is such an interesting question that extends beyond the article to consider concrete actions practitioners can take to develop assessment literacy. If we think of reflection as involving cycles of monitoring, planning, and critically evaluating one’s practices to gain enhanced awareness, develop plans for action, and consolidate new insights for further action, then it can play a critical role throughout the different stages of developing assessment literacy. For learning to be transformative, a concept often accredited to Mezirow’s influential work in the early 1990s, the approach encourages “a deep engagement with and reflection on our taken-for-granted ways of viewing the world, resulting in fundamental shifts in how we see and understand ourselves and our relationship with the world” (Journal of Transformative Education, 2020, n.p.). This orientation requires us to engage in critical reflection to examine our beliefs, values, assumptions, experiences, and understanding as they influence our assessment-related thoughts and practices. Reflection can propel development through increasing a practitioner’s control over his or her own learning and constructing knowledge and practices that promote transformative learning.
As the findings of the article made clear, assessment is something that all practitioners spend a great deal of their time and energy doing, yet rarely gets talked about. The call for open discussion and continued dialogue about assessment practices with other practitioners to share challenges and practices within our professional community dovetails with engaging in reflective practice. This may be mediated through reflective discussion about assessment practices with colleagues; written discussion with members of the professional community about discoveries, challenges, and strategies; or personal written or dialogue journals for recording experiences for various audiences (for oneself, to share with another teacher, to publish to inform a professional community, or to publish to a broader professional/scholarly community) (refer to Rathert & Okan, 2015).
SoTL Matters: What are some initial steps that can be taken to encourage collaborative development of assessment literacy between colleagues?
Prof Huang: One’s approach to developing assessment literacy collaboratively may depend on a confluence of factors, such as the following:
- Is there in-house expertise in assessment?
- What is the level of assessment literacy among colleagues?
- What is the degree of standardization of assessment within a unit?
- Is more than one instructor doing similar work or teaching different sections of the same course or a series of courses?
- What core testing- and assessment-related knowledge and skills are most needed in one’s particular instructional context?
- Does the institution allocate time and resources to support such collaboration for professional development?
Your responses to these questions may determine the best path(s) toward a shared goal. For those with in-house testing and assessment specialists, as some participants in the article reported, working with them can scaffold and support the collaborative process. Without the luxury of such specialists within a unit, look for inspiration within your unit and beyond by joining or forming PLNs (personal/professional/personalized learning networks) tailored to your own needs or to shared needs and interests. The initial steps may be as easy as getting together with a few colleagues who are doing similar work. A great starting place is to connect regularly to talk about what and how you go about assessing student learning and meeting institutionally mandated assessment demands, and the challenges you have encountered (e.g., assessment for teaching, accountability, or diagnostic purposes). You can then pinpoint what you might like to tackle first (e.g., basic concepts, refinement of assessment methods, development of assessment tools, validation of tests, etc.) and how to go about doing so. Seek out resources and/or reach out to contacts who might be able to mediate understanding or help problem-solve if the group gets stuck on a particular issue or area. As I previously shared, having a plan, a clear intent, and a way to sustain (Huang, 2015) is key to professional self-development for practitioners. If you have no one in your immediate circle to connect with, using social media (many engaged practitioners and researchers are on Twitter, for example) to find someone who shares a similar interest is another great way to connect and start the conversation.
SoTL Matters: How can practitioners and their institutions work together to develop assessment that promotes learning, rather than assessment that merely evaluates student work?
Prof Huang: Similar questions have been on the minds of many practitioners I have been fortunate to cross paths or work with. Trying to answer this question without thoughtfully taking diverse instructor-, learner-, institutional-, and contextual variables into account risks being overly simplistic in addressing how one can tackle such a reality as shared by many participants in the study and beyond. It is a challenging problem because one of the overarching goals—success in student learning—is shared by institutions and practitioners, but at the same time, as the article elucidated, practitioners and institutions often diverge in how they value or prioritize the purposes or functions of assessment. The issues of dual roles (instructional vs. evaluative) and dual functions (learning assessment vs. accountability assessment), as pointed out in the article, are thorny yet important to navigate because they have both a direct and indirect influence on what practitioners do in their teaching and, by extension, what students are taught or focus on.
The first question to ask may pertain to the degree of flexibility an institution provides to its instructors in determining how learning/performance is assessed. In my study, some instructors were given full freedom without oversight or support (a blessing and a source of great apprehension, anxiety, and fear at the same time), while other institutions sought to standardize the assessment methods and procedures. On a continuum, is assessment of student learning or performance more of a top-down approach (standardized to different degrees) or a bottom-up approach (where instructors get to determine the content and methods with varying degrees of supervision or support)? For the former, consider asking what feasible formative assessment components you can include to inform teaching and learning while keeping the objectives aligned with the curriculum’s intended outcomes as established by the program or institution. Are there in-house or professional development opportunities you can participate in to share some of your learning, and the outcomes of your students’ learning, to not only benefit other practitioners but also create institutional buy-in? This knowledge-mobilization step applies to the other end of the spectrum as well, where the outcomes of learners’ and instructors’ experiences can be made visible through sharing them with your intended audience.
If the institutional approach leaves no room for alternative forms of assessment, advocating as a group for integrating both formative and summative assessment would be another way to introduce assessment designed to inform teaching and promote learning. This also has the potential of providing one’s institution, if shared through impact-related SoTL research (Hubball & Clarke, 2010), different sources of data to evaluate learning or program effectiveness. One thing to consider is that assessment that merely evaluates student work can still be used to promote learning for some learners, while assessment that promotes learning can often still be used to gauge and show the developmental trajectories of learners. What can you do in your own instructional context to leverage the support practitioners need to experiment with different approaches to assessment in order to work toward bridging the potential mismatch of perspectives between practitioners and institutions?
I hope that the article and these responses to the questions have helped us to begin to peel back some of the layers and complexities of assessment. It is uplifting that the results of the research calling for critical dialogue has led to discussion among a group of dedicated SoTL practitioners, followed by this open exchange about an area of our work that often consumes practitioners’ thoughts, energy, and time, and where the approach is often top-down, or, if bottom-up, training and support is often lacking, as revealed in the study. Through this exchange, I hope that our colleagues’ scholarly approach to thinking about how to take action to improve our assessment literacy and practices has been acknowledged, and that practitioners’ perspectives and voices about the challenges of aligning teaching and assessment can continue to be amplified and heard in order to promote changes in our practice.
References
Huang, L.-S. (2015, Fall). Professional self-development for teachers: Have a plan, a clear intent, and a way to sustain. TEAL News. Retrieved from https://www.bcteal.org/wp-content/uploads/2017/12/BCTeal-Newsletter-Fall-2015-Final-2.pdf.
Huang, L. S. (2018). A call for critical dialogue: EAP assessment from the Practitioner’s perspective in Canada. Journal of English for Academic Purposes, 35, 70-84.
Hubbal, H., & Clarke, A. (2010). Diverse methodological approaches and considerations for SoTL in higher education. The Canadian Journal for the Scholarship of Teaching and Learning, 1(1), Article 2. Retrieved from https://ir.lib.uwo.ca/cjsotl_rcacea/vol1/iss1/2/?utm_source=ir.lib.uwo.ca%2Fcjsotl_rcacea%2Fvol1%2Fiss1%2F2&utm_medium=PDF&utm_campaign=PDFCoverPages
Rathert, S., & Okan, Z. (2015). Writing for publication as a tool in teacher development. ELT Journal, 69(4), 363—372.