Feedback literacy through diagramming
Feedback in design education is an interesting area… and when I say interesting, I think I probably mean something else.
Design tutors seem to have a lot to say about the volume of one-to-one tutorials that students demand, and the reams of written feedback they are expected to generate for assessment. Design tutors (often the same people) seem to have less to say about how we might re-imagine or re-design those very same feedback and assessment practices. There’s seems to be an underlying sense that however arduous feedback processes are, they must persist in their current form. That there is… no alternative.
Whose feedback?
To give a specific example, a colleague was once complaining to me about a lack of student particpation in crits. Apparently, this tutor needed to talk the ‘whole time’ in order to provide the ‘necessary’ level of feedback. Many tutors have experienced this phenomena; where learners engage with feedback as if it were solely an ‘input mechanism.’ (Barnett & Coates, 2005) I think many of us are keen to avoid tutor-led monologues, preferring that feedback be a space of dialogue. However, when I asked this colleague if they would consider designing the sessions in a way that modelled peer feedback for his students, their reply was really insightful.
“I hear what you’re saying” they said, “ the thing is, these students pay a lot of money for my feedback.” 😳
If you found yourself twitching at that comment, I’m with you. Sadly, it’s still quite common to hear things like this in subjects where the ‘transmission’ and ‘delivery’ of ‘expert’ feedback is the focus. On my more reasonable days, I would say that this is somewhat inevitable in disciplines that evolved from more vocational approaches. You can probably guess what I would say on my less generous days.
Such ‘transmission-based’ approaches are not only a bit old fashioned, they’re out of step with a growing body of research on feedback literacy. Carless & Boud (2018) have suggested that feedback literacy involves four inter-related features: appreciating feedback, making judgments, managing affect, and taking action. Their framework emphasizes the importance of a learners ability to:
understand and value feedback
critically evaluate it
regulate their emotions in response to feedback, and
use feedback to inform their future actions.
This is similar to Winstone et al., 2019 view that feedback literacy comprises cognitive, affective, and behavioral dimensions and can be conceptualised as three domains: knowing, being, and acting. The knowing domain is similar to understanding, valuing and evaluating in that it refers to learners' knowledge of the learning potential of feedback. The being domain relates to the impact of feedback on identity and emotion. The acting domain emphasizes the importance of taking concrete action in response to feedback.
The curriculum
The strong recommendation in all much of the literature is that feedback literacy should be embedded within the curriculum and it should support learners' to use feedback for their own learning (Malecka et al., 2020). It highlights the importance of providing opportunities for learners to elicit feedback, but also make sense of and apply feedback in their academic work.
Essentially, curriculum design should take an integrative approach that prioritises the learners evaluative judgment; their ability to make sound decisions about the quality of their own work and the work of others (Carless, 2020).
Nicol, 2010 also suggest that learners are more empowered when they can access feedback based on their own concerns. This is the opposite of our earlier example; a tutor doling out feedback on the basis of the weaknesses or strengths they perceive in the learners work.
The reasons that learners may not feel comfortable in bringing their own concerns into feedback spaces is a conversation we all seem a bit too eager to avoid. My own experiences would suggest that it has a lot to do with who and what is ‘centred’ in these moments of feedback. Spoiler: certain kinds of student, certain kinds of interaction and very often, tutors.
Not-feedback…
What we can take from all of this is one clear message…there’s no pedagogical justification for feedback if it does not influence learning. (Henderson et al. 2019) Sounds obvious? Yes. We all know that feedback influences learning on some level. I’m just not sure we can always be sure about how well it does this.
If you’ve ever found yourself at a summative assessment with the sense that the student you’re grading hasn’t ‘listened to a word’ of your feedback 👋🏻, chances are you’ve (I) been dabbling in a bit of ‘non-feedback.’
Feedback literacy advocates for centring student actions in feedback processes. It expands the definition of feedback to include feedback from tutors, peers and vitally, the feedback that learners generate themselves through self-assessment (Boud and Molloy 2013; Winstone et al. 2017). In such approaches the focus is less on the ‘delivery’ of feedback through ‘formats’ like crits and tutorials, than on the actual processes that support learners to make sense of feedback. Most importantly, how to make it work for them (Carless and Boud, 2018). This acknowledges the glaringly obvious fact that if learners can’t put feedback into a meaningful context, then it’s not really feedback.
A process of translation
In my own explorations of these ideas, my focus was on this notion of a ‘meaningful context.’ The meaning-making of feedback does not take place in a vacuum. Esterhazya & Damşab (2019) Feebdack is always embedded in a disciplinary context. One with it’s own knowledge, methods and practices. Reflective practice is widely acknowledged to play a vital role in design education. It is thought to enable learners to critically analyse their work and make informed decisions about it. This led me to question how the meaning-making of feedback might effectively take place through experimentation with disciplinary methods. How might we better use these disciplinary methods to make learning and self-reflection more explicit to learners?
As a Course Leader In Graphic Design I’m really interested pedagogical approaches that integrate diagramming. Diagrams work by visually relating different types of content. Diagrams are operational in that they help us to connect thinking and intuiting (Kramer, 2009). IMHO, this makes them great tools for thinking about thinking, and relating thinking to experience.
I’ve explored the knowledge that learner-generated diagrams bring into the curriculum before. Building on this, I found myself interested in the ways that diagrams could provide tangible and structured frameworks for reflection and feedback. How might diagrams practically support learners to think about their thinking? Could they use them to gain insight into their decision-making processes, understand their strengths and identify their areas for improvement? Could this thinking provide the basis for more meaningful feedback discussions?
What I did…
I presented some of my work on this at the recent AHE Conference in Manchester. I described a pattern I was noticing with learners transitioning between the two units of a one-year course. There were particular struggles developing self-directed projects. I discussed the ways I had used researcher-led diagrammatic elicitation (Umoquit et al, 2013), to enable externalization of learning (Boyd Davis & Vane, 2020) to support learners to make their internal feedback explicit (Nicol & McCallum, 2022) and enable a better quality dialogue between tutors, peers and learners.
In non-academic-paper-speak that is… I identified bottlenecks in the self-directed learner journey and designed a bunch of diagramming and data drawing exercises. These exercises supported learners to reflect on their learning, often in direct relation to the learning outcomes and assessment criteria. This improved feedback conversations, where learners tended to arrive with an understanding of where they were at and what they wanted to discuss. This in turn, appeared to vastly improve attainment between the first and second unit (when compared with the previous year). All the while, my students were experimenting with information design. Win, win.
Overall, the literature on feedback literacy emphasizes the importance of learners' ability to appreciate, evaluate, and use feedback effectively. It highlights the cognitive, affective, and behavioral dimensions of feedback literacy and suggests that it should be integrated into the curriculum to support learners' learning and development. This is, in essense, a design challenge and obviously, diagrmmaing is just one way of apporaching it. In sharing these experiements in practice at AHE this year, I wanted to open up discussions about the different ways we support feedback literacy through discplinary methods.
If you’re interested to see the diagrams, the impact on attainment and the student feedback you can see them in the AHE Conference presentation. There’s obviously way more to the design of this than I presented. That’s because conference sessions are very short and curriculum-based action research is… well, not.
I put together another presentation to the Illustration programme in the Design School on how all this worked, in eye watering and painstaking detail. Happy to share that with anyone who is genuinely interested. You can also see the workshop slides from a CPD session I ran for schools staff on how I actually designed the diagrams. In case you’re interested in adapting this for your own context.
And of course, you’re always welcome to get in touch. Apparently I just love talking about diagrams.
With thanks to…
Boyd Davis, S., & Vane O. (2020) Design as Externalization: Enabling Research, Information Design Journal 25 (1): 28–42
Boud, D., & Molloy, E. (2013) Rethinking models of feedback for learning: the challenge of design, Assessment & Evaluation in Higher Education, 38:6, 698-712
Carless, D., and D. Boud. 2018. The Development of Student Feedback Literacy: Enabling Uptake of Feedback, Assessment and Evaluation in Higher Education 43 (8): 1315–1325. doi:10.1080/ 02602938.2018.1463354.
Carless, D. (2020). From teacher transmission of information to student feedback literacy: activating the learner role in feedback processes. Active Learning in Higher Education, 23(2), 143-153.
Dawson, P., Carless, D., & Pui Wah Lee, P. (2021) Authentic feedback: supporting learners to engage in disciplinary feedback practices, Assessment & Evaluation in Higher Education, 46:2, 286-296,
Drucker, J. (2014). Graphesis : visual forms of knowledge production. Harvard University Press, Cambridge, Massachusetts.
Esterhazy, R. and Damşa, C. (2019) ‘Unpacking the feedback process: an analysis of undergraduate students’ interactional meaning-making of feedback comments’, Studies in Higher Education, 44(2), pp. 260–274.
Henderson, M., R. Ajjawi, D. Boud, and E. Molloy. 2019. “Identifying Feedback That has Impact.” In The Impact of Feedback in Higher Education: Improving Assessment Outcomes for Learners, edited by M. Henderson, R. Ajjawi, D. Boud, and E. Molloy, 15–34. Basingstoke: Palgrave Macmillan.
Hutchins, E. (1999). Cognitive artifacts. In R. A. Wilson & F. C. Keil (Eds.), MIT Encyclopedia of the Cognitive Sciences (pp. 126–127). Cambridge, MA: MiT Press.
Krämer, S. (2009) Epistemology of the line’ Reflections on the diagrammatical mind, Freie Universität Berlin
Laurillard, D. (2002). Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. 10.4324/9780203160329.
Laurillard, D. (2012) Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York: Routledge
Malecka, B., Boud, D., & Carless, D. (2022) Eliciting, processing and enacting feedback: mechanisms for embedding student feedback literacy within the curriculum, Teaching in Higher Education, 27:7, 908-922, DOI: 10.1080/13562517.2020.1754784
Middendorf, J. & Pace, D. (2004), Decoding the disciplines: A model for helping students learn disciplinary ways of thinking. New Directions for Teaching and Learning, Vol 2004:98
Molloy, E., Boud, D., & Henderson, M. (2020) Developing a learning-centred framework for feedback literacy, Assessment & Evaluation in Higher Education, 45:4, 527- 540
Nieminen, J (2022) Disrupting the power relations of grading in higher education through summative selfassessment, Teaching in Higher Education, 27:7,892-907
Nicol, E & McCallum, S (2022) Making internal feedback explicit: exploiting the multiple comparisons that occur during peer review, Assessment & Evaluation in Higher Education, 47:3, 424-443
Nicol, D (2010) From monologue to dialogue: improving written feedback processes in mass higher education, Assessment & Evaluation in Higher Education, 35:5, 501-517
Price, M. et al (2012) Assessment Literacy: The Foundation for Improving Student Learning. OCSLD: Oxford.
Sutton, P. (2012). Conceptualizing feedback literacy: knowing, being, and acting. Innovations in Education and Teaching International, 49(1), 31-40.
Umoquit, M., Tso, P., Varga-Atkins, T., O'Brien, M., & Wheeldon, J. (2013). Diagrammatic Elicitation: Defining the Use of Diagrams in Data Collection. The Qualitative Report, 18(30), 1-12.
Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017). Supporting learners' agentic engagement with feedback: A systematic review and a taxonomy of recipience processes. Educational Psychologist, 52(1), 17–37.
Winstone, N., Mathlin, G., & Nash, R. A. (2019). Building feedback literacy: students’ perceptions of the developing engagement with feedback toolkit. Frontiers in Education, 4.