Dr Ritta Husted (International Academy) and Mr Kosin Panyaatisin (Language and Linguistics) report on the initial findings of a small-scale research project undertaken as part of TALIF project focused on foundation students’ engagement with written tutor feedback.
Feedback Really Matters
It is generally acknowledged that students’ mindful engagement with tutor feedback is more likely to lead to an increased sense of ownership of work, deeper subject understanding and reflective practices (Price et al. 2010, Handley et al, 2011). For the purpose of this report, mindful engagement refers to providing students with opportunities for engaging reflectively with their written feedback and entering into a dialogue with the tutor (Scott et al 2011). The report will furthermore tentatively set out the relationship between the quality of tutor feedback resulting in a surface or deep approach to learning with the students’ active engagement and feed forward opportunities and provide some practical suggestions as to how to encourage mindful and active engagement by way of a Tutorial Journal and Feedback Matters assignment coversheet.
Handley and colleagues (2011) depict diagrammatically a typical feedback process allowing for the inclusion of students’ active engagement.
Figure 1: contextual and recursive influences on student engagement with assessment-and-feedback
(Handley et al 2011:550)
Weaver’s (2006) study identified that ‘students may need advice on understanding and using feedback before they can engage with it’ (Weaver, 2006:379). Out of 44 the university students investigated, 50% reported that they had not received any guidance on how to read feedback nor how to use it; Weaver suggests that it is plausible that a ‘sizable minority’ of university students in general might not be able to effectively address any identified gaps between current level of performance with that which is expected simply because they have been unable to act mindfully on feedback given. Similar sentiments are voiced by Quinton & Smallbone (2010) who suggest that ‘students need help in making the connections between their feedback, the characteristics of their work, and how to improve it in the future’ (Quinton & Smallbone, 2010:127).
The Feedback Matters Study
The study was two-fold: the first part examined 94 foundation year students from the International Academy at Essex during the academic year 2014-15 who were taking the IA140 Academic Skills module. It is this part which will be dealt with in some detail below. The second part examined eight foundation year students on the Health & Science pathway and eight foundation year students on a Social Science pathway. Three distinct modules were chosen in order to capture the diversity of assessment methods and products: in-class test/essay writing/report writing on IA140 Academic Skills, IA118 Politics & Sociology and IA160 Computer Programming respectively. The second part of the study will be reported on in more detail once the analyses have been completed, however, some tentative conclusions will be put forward here.
Upon receiving written tutor feedback on their coursework and prior to attending either a group or a one-to-one tutorial, the students were instructed to review their annotated submitted coursework in conjunction with the tutor’s formative feedback and fill in a Tutorial Journal as part of the preparation for the tutor-student dialogue (equating to step 4 in Figure 1 above). This aspect of the feedback cycle is very much aligned with the findings presented by Weaver (2006) above, namely that students are likely to need some guidance on how to decipher the tutor message. It also resonates with the message from Quinton & Smallbone (2010) about the student’s need for opportunities to explicitly reflect on their feedback. Elements students were asked to consider for IA140 Academic Skills were:
- Based on the written feedback and comments my AS tutor gave me for my EAP Test, I found the following feedback and comments in particular helpful;
- I plan to pay more attention to the following aspects of my writing and feed them forward (incorporate them) into my summer exams;
- I would like to receive more clarification on the following comments.
( A full version of the Tutorial Journal can be found below.)
How did participants respond?
During a small focus group discussion in the Summer Term 2015 students from the IA140 Academic Skills module were asked to comment on the usefulness of the Tutorial Journal. They unanimously agreed that the journal helped them focus their reflections on the feedback and allowed them to identify any gaps in knowledge, or skills in this case. This was in particular in relation to question 2 above. They liked the fact that ‘it [the journal] was “straightforward” to use, very clear, and that it helped focus the tutorials’. They also mentioned that although they would be happy to use the Tutorial Journal but suggested that it would be even more accessible if the wording could be different and more emphasis could simply be placed on identifying your strengths and / or weaknesses.
Tutors’ feedback on using the Tutorial Journal was in general positive. They liked the idea that students arrived prepared to the tutorials and that students were encouraged to take a lead in the discussion around their feedback. It was perceived that quite often tutorials could end up being a verbal repetition of the written comments with none or minimal active engagement on the part of the students. One tutor had tried out the preparation of the Tutorial Journal stage in a group setting thus allowing class time for the students to carry out their reflections. Upon completion of their own reflective comments in the Tutorial Journal, they were then encouraged to share their identified strengths and / or weaknesses with each other; when they were asked about the usefulness of peer discussions, the overwhelming response was that it was helpful.
The vast amount of qualitative data collected allowed us to measure type and frequency of key words using ANTCONC. The students’ written reflective responses were grouped into five distinct areas: (1) mechanics of writing (including punctuation, spelling, pre-planning); (2) organisation (including structure, aim/outline and purpose for writing); (3) praising (including raising awareness of mistakes made); (4) referencing (including quotations and bibliography); and (5) writing style (including grammar, formality of writing / style, paraphrasing). The motivation for this particular division was in order to ensure alignment with the marking criteria adopted for this skills module; furthermore, within the wider context of EAP teaching and instruction these groupings do not seem illogical in a typical feedback setting.
Qualitative analysis
The qualitative data collection allowed us to measure the number of tokens of salient key words assigned to each of the five groups; we were then able to measure the correlation between questions 1 and 2 above where we found no significant difference at p-value >0.5. In this case, this is not surprising as we were hoping that reflections adhered to in question 1 were paralleled in question 2.
| Table 1: t-Test: Two-Sample Assuming Unequal Variances | ||
| Variable 1 | Variable 2 | |
| Mean | 34.6 | 34.4 |
| Variance | 393.3 | 364.3 |
| Observations | 5 | 5 |
| Hypothesized Mean Difference | 0 | |
| df | 8 | |
| p-value | 0.987435 | |
| t-test score | 2.306004 | |
The findings can be seen in Figure 2 below.
Figure 2: The distribution of word frequencies from Question 1 and Question 2 across types of tokens
These initial findings suggest that students do indeed engage actively with the feedback when opportunities and guidance are made available. They also suggest that students are able to reflect mindfully on the tutor feedback provided by aligning tutor comments with points for feeding forward.
The students were also encouraged to include their reflective comments from the Tutorial Journal in the ‘Reflections’ box on the Feedback Matters assignment coversheet (see link below) as a way of informing the tutor of their reflections; especially in cases where students were feeding forward to an assignment on a different module.
Subject module findings
The second part of the study looked at the student engagement with feedback from subject modules as opposed to the skills module described above. This was in order to be able to determine what particular aspects of tutor feedback students reflected on; e.g. surface comments which would relate to the mechanics of the work such as: essay organisation, language, referencing and punctuation or on the other hand, deep comments which would relate to: content, examples included, and strength of the argument. In order to determine what kind of tutor feedback the students attended to, it was necessary to include the tutors’ written feedback for analysis. The feedback process again involved the students reviewing their annotated submitted coursework in conjunction with the tutors’ formative feedback and fill in a Tutorial Journal in preparation for their tutorials. Elements students were asked to consider for IA118 Politics and Sociology and IA160 Computer Programming were:
- Based on the feedback / comments my tutor gave me, I plan to pay particular attention to the following aspects and feed them forward (incorporate them) into my next written assignment;
- Apart from the feedback / comments the tutor gave me, I would like to receive more feedback on the following aspects of my work next time
Early findings indicate that the students did engage actively with the feedback provided. So, the answer to Higgins’ et al (2002) question: do students take notice of feedback? The answer must be ‘yes’. The findings also suggested that the students’ reflections were primarily driven by their tutors’ feedback. For example, in instances where a tutor primarily focused his or her feedback on surface elements including those described above, the students would also focus on these aspects. However, in cases where a tutor had primarily commented on subject content, also referred to as deep elements, the students reflected on these. These findings will be analysed in more detail using ANTCONC in order to reveal the exact relationship between tutor comments and student reflections.
Interim conclusions
To conclude, it seems to be the case that students do engage mindfully with the feedback when done so in conjunction with an academic. It therefore seems to be the case that guidance and opportunities for dialogue are important aspects of the reflective feedback process and indeed help with making connections across different assignment opportunities. It also seems to be the case that the students’ reflections are driven, almost entirely, by the tutors’ comments, at least at foundation level study, suggesting that the quality of the feedback is critical. It would be interesting to see if early training of students in reflective practices could in time become an automated (transferable) skill which would help foster independence and empower students.
Dr Ritta Husted
Mr Kosin Panyaatisin
Bibliography
Handley, K, Price, M and Millar, J (2011). Beyond ‘doing time’: investigating the concept of student engagement with feedback. Oxford Review of Education, vol 37, no 4, pp. 543-560
Hattie, J and Timperley, H (2007).The Power of Feedback. Review of Educational Research. Vol 77, no 1, pp. 81-112
Higgins, R, Hartley, P and Skelton, A (2002). The conscientious consumer: reconsidering the role of assessment feedback in student learning, Studies in Higher Education, vol 27, no 1, pp. 53-64
Hyland, F (2003). Focusing on form: student engagement with teacher feedback. System 31, pp. 217-230
Krause, K (2005) Understanding and promoting student engagement in university learning communities. CSHE. Paper delivered at James Cook University, September 21-22, 2005.
Quinton, S and Smallbone, T (2010). Feeding forward: using feedback to promote student reflection and learning – a teaching model. Innovations in Education and Teaching International, vol 47, no. 1, pp. 125-135
Scott, J, Shields, C, Gardner, J, Hancock, A and Nutt, A (2011). Student Engagement with Feedback. Bioscience Education, Vol. 18, DOI: 10.3108/beej.18.5SE
Trowler, V and Trowler, P (2010). Student engagement evidence summary. The Higher Education Acadmey, June 2010
Weaver, R M (2006). Do students value feedback? Student perceptions of tutors’ written responses. Assessment & Evaluation in Higher Education, vol. 31, no. 3, pp. 379-394
