Audio assessment and student learning

 

Keywords: assessment, feedback, audio, reflection, motivation

Abstract

Our project was designed to test the potential of audio feedback as a constructive and motivating aspect of student learning within a practice-based course. We looked into the form, content and timing of the feedback, and at both the immediate and subsequent responses of students in relation to their learning and motivation. We were also interested in the extent to which audio might provide the type of personal and supportive discursive feedback offered by the traditional studio crit – particularly where pressure of time and numbers makes this option problematic.

Rationale

Assessment feedback is intended to provide a motivating aid to reflection and learning. However, experience and student feedback suggests that the reality is very different and that assessment, whether formative or summative, frequently inhibits the very qualities of reflective understanding that it is intended to support. We were concerned by the tendency of students to lose confidence and motivation in the face of a disappointing grade - even one which was swiftly, accurately and clearly conveyed to them. We were also aware that tutor comments were often ignored, particularly by grade-fixated final-year students. In short, we felt that our assessment strategy was no longer effective and that there was a severe and increasing imbalance between the amount of staff time taken up by the activity and the learning outcomes for students.

Context

Our action-research project was part of a general review of assessment on a 3-year BA Graphic Communication programme in Cardiff School of Art & Design (CSAD), University of Wales Institute, Cardiff (UWIC). We worked with first and third-year cohorts. Evidence was gathered during a single academic year and across a range of modules, with assistance from full-time and part-time tutors.

In years one and two the existing assessment scheme officially focuses on diagnostic feedback: none of the modules completed at this stage affect the final degree award. Students hand-in or present work at the end of a module and receive an alphabetic grade and written feedback based on the stated assessment criteria. Oral feedback is provided throughout modules with individual and group tutorials led by both staff and students. Our particular concern in year one and two was that relatively few students appeared to be using either their continuous or final assessment feedback as a diagnostic aid to reflection and learning.

In year three the basic format of assessment remains as in years one and two. However, at this stage all modules count towards the degree classification, leading to a much higher level of assessment anxiety among many students, and a related decrease in motivation if they receive a ‘low’ grade. In an attempt to ameliorate this problem the year three scheme does acknowledge ‘exit velocity’ by providing students with the opportunity (limited by institutional regulations) to rework and represent module outcomes for regrading at an end-of-year portfolio presentation.

Aims

Our ambition was to maximize student use of assessment feedback for their own development, and to transform assessment into a more supportive, personal and motivating aspect of programme delivery.

Description of project

We used audio assessment feedback with 31 third-year students and 26 first-year students from the BA Graphic Communication programme at CSAD. The audio feedback was in addition to the timetabled individual and group crits which take place in the studio. Individual audio assessments were between 5-10 minutes in length. Staff simply spoke to the work in relation to the assessment criteria with the commentary preceding the grade. Comments were individual, detailed and supportive, with a focus on ways in which the work could fulfil its potential. No solutions were provided, but perceived weaknesses in the work were discussed and possible directions suggested along with a rationale for the tutors’ opinion. The feedback was sent out via MP3 files within 48 hours of the project hand-in, and generally within 24 hours. Files were not edited, partly due to time constraints but also since we wanted to maintain a sense of discursive informality, a sense that this was a conversation, albeit with only one speaker.

A simple, inexpensive, hand-held recording device was used; a quiet space was the other pre-requisite. Five members of staff took part in the initial research project which involved a series of interventions with the year one and year three cohorts.

Year one interventions

Students were introduced to the assessment system during the first week of their first term. We discussed the problematic aspects of both form and timing of traditional post-hand-in assessment, and explained the intentions of the trial system. Then, during the term-long module, students were sent audio assessment feedback and potential grades (via an MP3 file) on three occasions, each relating to a specific project. A reflection/reworking period of one week was timetabled at the end of term prior to the formal module hand-in. Students were then given a summative grade and a very brief written commentary as hard-copy.

Year three interventions

To lessen the likelihood of our research intervention adding to student anxiety, we used a two-stage approach. Stage one was used to compare student responses to audio assessment vs written assessment. Stage two was used to evaluate the capacity of audio assessment to encourage motivation and reflection.

Stage one

At the end of a module we gave all students graded audio feedback via an MP3 file instead of the more usual written feedback. Comments preceded the grade and the general approach was as for the year one intervention. All other aspects of the third-year assessment system remained unchanged.

Stage two

The following term we gave all year three students audio feedback and a potential grade one week before the end of a module. Students were encouraged to reflect on their project through this feedback and to work further on it for the final hand-in. They were then given a final grade with a one-line written comment.

Evaluation

We collected quantitative data relating to perceived usefulness, quality, effect on motivation and on engagement via student questionnaires which also allowed for qualitative comments. More in-depth, discursive data was provided by a focus group of final-year students, and through informal discussions with staff who had taken part in the project. Further evidence for the capacity of the audio feedback to encourage reflection and motivation was provided by the increased quality of module outcomes and the overall improvement in the grades awarded.

In general the response of staff and students involved in the action research was hugely positive, both in relation to the use of audio assessment in its own right, and to the inclusion of a 'reflection' period after the audio feedback and before the final, graded assessment point. Significantly, the change from extensive written commentary to the ‘one-liner’ was not viewed as problematic: students and staff saw this change of emphasis from summative to formative as both appropriate and useful.

Specifically:

1. Audio assessment offers a more personal and pleasurable experience

Both staff and students were very positive about giving and receiving feedback in an audio form: staff admitted to actually enjoying the experience and several students spontaneously emailed tutors to thank them for the feedback – a new experience for us! Even allowing for the student who was rather disturbed by the sound of a tutor’s voice booming round his bedroom, the student response was consistently positive:

• ‘Nice to hear a voice, hear the tone of the voice, more personal.' 
• 'Audio feedback is much more inviting than written feedback.'
• 'Left me feeling positive whereas with written feedback I don't tend to read it again if I've done badly.'

2. Audio assessment encourages engagement and learning

Students listened to their audio assessment more thoroughly and more frequently than they read their written feedback; their responses suggest that their reception of audio feedback is more open-minded and engaged:

• ‘When it was delivered it was as if someone was speaking to me, it felt like it was much more important, it really registered whereas when you get paper, it doesn’t connect with me at all. It’s just a piece of paper, you can throw it away if you don’t like it but I made myself listen to it again, to really pay attention to what you said about my work because it does make me really want to improve.’
• 'I think I pretty much remember everything that was said to me on the audio as opposed to something that’s written down. You’re more likely to remember something you’re told, it’s the way you learn isn’t it.'
• '[The audio is] much clearer to understand. Friendlier. Thorough. It has encouraged me to rework my ideas.'

3. Audio assessment encourages a focus on the diagnostic function of feedback

The personal and engaging quality of audio assessment encouraged students to view the feedback as supportive rather than critical. They were more inclined to focus on the diagnostic commentary rather than becoming fixated on the grade:

• 'It was better feedback - had more of an impact. You had to listen more to what was being said rather than just your grade. It made you focus on things about your actual project.’
• 'Because you [the tutors] had more free flowing time to talk about [the work], for personal development as a designer that is a lot easier to take on. There are a lot more points to take away. [Written] feedback sheets, if you are rushing to get them back to us, there are only two or three points on there. You end up thinking I need more clarification as to why I haven’t got a better grade, whereas with the audio feedback, there was a lot of stuff in there.'
• 'It's so much easier to improve the work [with audio]: with a feedback sheet you read it and take away what you have got from it and then kind of leave it for a while, whereas with the audio you can go back to it and listen to it again while you are redoing it.’

4. Providing interim audio assessment prior to the 'reflection and reworking' period ensures that it becomes a genuine part of the learning process

On the basis of this research, audio assessment would appear to be more effective than written assessment in its own right. However, its usefulness as an aid to reflection is hugely increased when students are given the opportunity to use the feedback to improve their work before any summative assessment takes place:

• 'My project changed so much since the first audio feedback, because that was the interim assessment and I had a whole week to change it and it really did change a lot.'
• 'I dreaded the graded one [year three stage one] but I knew that the interim one was going to help so I was much more open to listening to it.'
• 'It was a more positive experience than the previous [stage one audio assessment]. I listened to it a lot more because everything you were saying, I wanted to write it down so that I could change it, and then I’d be playing it again. It wasn’t like listening to it and then at the end of it you’d feel down or whatever because you know that you’re just saying ‘right do this to improve it’ and you’re like ‘right okay yeah’. It was just like having a tutorial.’

These responses encouraged us to formally modify the assessment system for the programme. Reflection and portfolio preparation periods are now timetabled in for all three years and summative assessment occurs after these opportunities for the majority of modules. We have the same number of summative assessment points, but written feedback is minimal; tutors’ time and energy is focussed on the diagnostic stage of the process, making it a more meaningful and enjoyable experience. We have found that two members of staff can complete up to 50 assessments in one (busy) day, spending the first hour benchmarking the work, and then completing between 5-8 individual assessments in an hour. It is not necessarily a speedier process, but it is significantly less tiring and more engaging for staff.

For all participants, the combination of audio assessment and a reflection period have significantly helped to reposition assessment as a positive and purposeful teaching and learning strategy. Most significantly perhaps, the approach has reaffirmed the staff-student partnership which is at the heart of education, turning a pedagogic aspiration into an educational reality.

Biographies

Ruth Dineen is a Teaching & Learning Associate at UWIC and was previously Head of the Department of Creative Communications at Cardiff School of Art & Design. She has a particular interest in creative pedagogy, student motivation, and assessment for learning. Ruth is currently using her National Teaching Fellow (NTF) award to look at the possibility of co-producing a cross-sector co-production toolkit for practitioners. She is on a steep learning-curve!

Annie Grove-White, formerly a Principal Lecturer and Director of Student Development at Cardiff School of Art & Design, is currently a visiting lecturer at Arts University College Bournemouth as well as a consultant and Action Learning facilitator in Higher Education. She is a National Teaching Fellow of the University of Wales and her passion and research interests are in the area of enriching the student experience. Her recent research, in partnership with Kirsten Hardie, investigated how an undergraduate research symposium and in particular, the role of the poster, can enrich the academic experience of final year students.

References

In terms of the practical aspects of audio assessment, we found the following links/references very useful:

Sounds Good project: http://www.soundsgood.org.uk

Middleton, A. & Northcliffe, A. (2010) 'Audio Feedback Design: principles and emerging practice'. International Journal of Continuing Engineering Education and Life-long Learning, 20:2, pp.208-223.

Raven, D. (2008) 'CLIP CETL Verbal Feedback Project Report' in Effective Practice in a Digital Age: case studies, http://www.jisc.ac.uk/resourceexchange.

For a broader, contextualised view:

Winter, R (2003) 'Contextualising the Patchwork Text: addressing problems of coursework assessment in higher education', Innovations in Education and Teaching International, 40:2, pp.1-21.

Other references:

Barnett, R. (2007) ‘Assessment in Higher Education: An impossible mission?’. In D. Boud and N. Falchikov (Eds.) Rethinking Assessment in Higher Education: learning for the longer term, London, Routledge. 

Gibbs, G. & Simpson, C. (2004) 'Conditions under which assessment supports students' learning', Learning & Teaching in Higher Education, 1, 2004-5.

Ramsden, P. (1992) Learning to Teach in Higher Education, London, Routledge.

 

Header and listing photo: sourced from morgueFile.com