Using audio for feedback on assessments : tutor and student experiences

Recently we have been providing individual audio feedback to 1 and 2 year undergraduate Chemistry students on a variety of assessments (posters, laboratory reports, laboratory diaries) with the aim of providing richer, more detailed and more comprehensible individual feedback than is possible within the same timeframe using written feedback. In this communication, various aspects of the use of audio for feedback are discussed including practical and technical aspects of the recording of audio files whilst viewing and assessing student work, the transmission of these files to individual students, our experiences as tutors of providing audio feedback and the experiences and views of students on audio feedback. Introduction Evidence from the National Student Survey continues to suggest students are not satisfied with their experiences of assessment and feedback in UK HE. Specifically, in terms of feedback on assessments, the timeliness (Feedback on my work has been prompt), level of detail (I have received detailed comments on my work) and comprehension (Feedback on my work has helped me clarify things I did not understand) of feedback attracts consistently low scores. The prevalence, accessibility and affordability of digital technologies (digital audio, screencasting, webcams etc.) offer new opportunities and possibilities in teaching and learning and specifically in the enhancement of the level of detail and comprehension of the feedback possible through the rich palette of the voice and the shorter time taken to speak comments compared with the time taken to write them. The use of audio for feedback and recognition of its potential merits is, however, not new, but prior to the digital age it was not widely adopted, at least in part because of the practical difficulties associated with the technology of the time. Although still relatively uncommon at the time of writing, it is evident that the use of audio for feedback is increasing in UK HE and it is emerging as an attractive and convenient alternative to handwritten or typed feedback on assessments. In comparison with handwritten/typed feedback, digital audio offers tutors an accessible and convenient means for providing richer, more detailed and more comprehensible feedback to students on their work without it taking more tutor time (and perhaps saving time). Nuances can be conveyed through tone of voice and use of language that would simply take too much time to achieve in written feedback. Specifically, in comparison with handwritten feedback, legibility is not an issue. With today‟s heavy workloads and considerable time pressures, it is also far more tempting to curtail detail in written feedback to students than is the case with recorded spoken feedback. This communication recounts some of our experiences of providing audio feedback on various undergraduate assessments in chemistry as well as the students‟ experiences of receiving feedback via audio. General Technical Aspects Initial trialling of audio feedback employed a laptop with an internal microphone and freely available software 8 enabling the production of mp3 files. However, this was rapidly succeeded by the use of hand-held digital mp3 recorders (with or without clip-on microphones) equipped with retractable USB ports and costing ~£50. Filenames included the student‟s name (and sometimes the actual piece of work where this varied between students in the same class). A clear structure to the audio feedback is useful and a prompt sheet may be helpful in this regard. File sizes were typically 1.3 MB per minute, although it is possible to reduce this. Audio files (mp3) were typically 5-6 minutes in duration and were returned to students individually via the VLE (WebCT) using a dummy „assignment‟. Using audio for Feedback on assessments: tutor and student experiences Evidence from the National Student Survey continues to suggest students are not satisfied with their experiences of assessment and feedback


Introduction
Evidence from the National Student Survey 1 continues to suggest students are not satisfied with their experiences of assessment and feedback in UK HE.Specifically, in terms of feedback on assessments, the timeliness (Feedback on my work has been prompt), level of detail (I have received detailed comments on my work) and comprehension (Feedback on my work has helped me clarify things I did not understand) of feedback attracts consistently low scores.The prevalence, accessibility and affordability of digital technologies (digital audio, screencasting, webcams etc.) offer new opportunities and possibilities in teaching and learning 2 and specifically in the enhancement of the level of detail and comprehension of the feedback possible through the rich palette of the voice and the shorter time taken to speak comments compared with the time taken to write them [3][4][5][6] .The use of audio for feedback and recognition of its potential merits is, however, not new 3 , but prior to the digital age it was not widely adopted, at least in part because of the practical difficulties associated with the technology of the time.Although still relatively uncommon at the time of writing, it is evident that the use of audio for feedback is increasing in UK HE [2][3][4][5][6] and it is emerging as an attractive and convenient alternative to handwritten or typed feedback on assessments.In comparison with handwritten/typed feedback, digital audio offers tutors an accessible and convenient means for providing richer, more detailed and more comprehensible feedback to students on their work without it taking more tutor time (and perhaps saving time).Nuances can be conveyed through tone of voice and use of language that would simply take too much time to achieve in written feedback.Specifically, in comparison with handwritten feedback, legibility is not an issue.With today"s heavy workloads and considerable time pressures, it is also far more tempting to curtail detail in written feedback to students than is the case with recorded spoken feedback.This communication recounts some of our experiences of providing audio feedback on various undergraduate assessments in chemistry as well as the students" experiences of receiving feedback via audio.

General Technical Aspects
Initial trialling of audio feedback employed a laptop with an internal microphone and freely available software 7, 8 enabling the production of mp3 files.However, this was rapidly succeeded by the use of hand-held digital mp3 recorders (with or without clip-on microphones) equipped with retractable USB ports and costing ~£50.Filenames included the student"s name (and sometimes the actual piece of work where this varied between students in the same class).A clear structure to the audio feedback is useful and a prompt sheet may be helpful in this regard.File sizes were typically 1.3 MB per minute, although it is possible to reduce this.Audio files (mp3) were typically 5-6 minutes in duration and were returned to students individually via the VLE (WebCT) using a dummy "assignment".2: Summary of responses to Q1-4 (see Figure 1)

Audio Feedback on Laboratory Diaries
Systematic assessment of laboratory diaries is common to most Chemistry modules at Keele and students are guided to include page numbers and to maintain a contents page from the very beginning of the degree programme, which facilitates the provision of feedback (written/typed or audio).Our first use of audio for feedback on laboratory diaries was in March 2010 in a 15-credit 2 nd year physical chemistry module (~36 students) involving practicals on electrolyte solutions and equilibrium electrochemistry, for which the laboratory diary comprised 15% of the module mark.Laboratory diaries were submitted mid-semester and returned with feedback and marks within 2 weeks.We have also recently provided (December 2010) audio feedback on laboratory diaries for ~100 1 st year students on a general chemistry module, but the main focus of what follows is on the 2 nd year laboratory diaries.

Practical Aspects
The provision of audio feedback on laboratory diaries was relatively straightforward to implement.The general procedure adopted was to read the diary and highlight areas for comment using a highlighter pen (short comments/words (e.g."units") were occasionally added as prompts for the subsequent audio feedback).In recording the feedback, each audio file included an introductory guidance comment to the student similar to the following: "This is the feedback on your lab diary for CHE-XXXXX.I"ve highlighted specific points in your lab diary to which my comments refer, so you will find it more useful if you listen to this feedback with your lab diary in front of you".
Following some introductory general feedback, the student is directed towards each specific section/area for feedback by referring to the page numbers and the highlighted sections/ areas.The conclusion of the audio file included a short summing up followed by the marks for the various assessed components of the diary and the overall mark.

Student Feedback
Of the 36 students 2 nd year students who received audio feedback on their laboratory diaries 21 (58%) completed a questionnaire (see Figure 1).A summary of responses to Q1-4 is provided in Figure 2.
Using audio for feedback on assessments: tutor and student experiences

Communication
The novelty of audio feedback to students is clear as no students (in this survey and others) said they had previously received feedback in this way.In this particular group of students 100% of respondents said they had listened to their audio feedback with 81% listening to the feedback with their work in front of them and 62% indicating they had listened to the feedback more than once.Typical reasons given for listening to the feedback more than once (~2/3 of respondents cited a reason) are listed below: Only a few students answered Q7 to suggest the inclusion of video or to conduct the feedback face-to-face (it was not stated that this should be captured in a recording, but this is the implication and is an idea that has been reported recently in the context of recorded personal feedback conversations in the laboratory 6 ).
For Q8 (would you have preferred an alternative form of feedback?),52% of respondents cited face-to-face feedback, but only 5% of students cited written feedback.
Similar themes arose with the feedback from 1 st year students, although the enhanced comprehensibility of audio feedback was more clearly evident in these students" comments.This cohort of students was specifically asked to explain whether they preferred audio or written feedback.Issues of accessibility and navigation of the audio feedback problems (see below) were particularly prominent in these students" comments:

Tutor Evaluation
From a practical point of view, the use of audio for feedback is ideally suited to the complexity and variety of work contained within a laboratory diary.It is comfortable to browse through a laboratory diary and to speak rather than write comments, with the only work your fingers are engaged in being the turning of pages and depressing the pause button on the digital voice recorder every so often.However, interruptions whilst recording audio feedback are more disruptive than for writing feedback, simply because you may lose your train of thought and you cannot "see" what you have previously said so readily.The student feedback is highly positive and the increased detail in the feedback provided using audio rather than written feedback is borne out by the students" comments.The principal drawbacks cited by students appear to be related to ease of accessibility and navigation.
Using audio for feedback on assessments: tutor and student experiences

New Directions
Communication Audio Feedback on Laboratory Reports Laboratory reports come in a variety of formats, but the reports for which audio feedback was provided in this work are formal structured word-processed (1000 words) reports, submitted and returned electronically via the VLE.Of course, there is no reason why feedback in audio format cannot be provided on hard-copy reports and assignments in a similar manner to that described for laboratory diaries above.

Practical Aspects
Audio feedback on laboratory reports was provided for two groups of students in 2010-11; 1 st year chemistry (entire cohort of ~100 students, December 2010) and 2 nd year chemistry (~50% of cohort (~30 students), March 2011, with the other ~50% of students receiving typed feedback from another tutor).In each case laboratory reports were word-processed (1000 words) and submitted online via the VLE.Marked laboratory reports with marks and embedded comment numbers (see Figure 3) and mp3 files were delivered to students individually within 2-3 weeks (2 nd year) and 4 weeks (1 st year).In recording the feedback, each audio file included the following introductory guidance comment to the student.
"This is the feedback on your lab report for CHE-XXXXX.I"ve placed comment numbers throughout your report and I will refer to these throughout this recording, so you will find it more useful if you listen to this feedback with your work in front of you".
The procedure adopted for assessing and providing feedback on laboratory reports was to read the report on-screen whilst simultaneously adding blank comments (occasionally words or short phrases were added as prompts for the audio feedback) using the comments facility within the review tab in MS Word.During this process marks were also assigned for the various aspects of the lab report against the assessment criteria (marks were provided in a table within the word document).This part of the process took typically 15 minutes.The audio feedback was then recorded using the comment numbers as navigational signposts for the students.The conclusion of the audio file included a short summing up and some specific advice on how the feedback could be used in future assignments (e.g. research project dissertations).

Student Feedback
Similar themes and issues encountered with audio feedback on laboratory diaries also arose with audio feedback on laboratory reports.Some representative feedback comments are provided below.The cohort of 1 st year students was specifically asked to explain whether they preferred audio or written feedback, whilst 2 nd year students were not specifically asked about audio feedback and their comments are unprompted within a module evaluation questionnaire (~30% of respondents highlighted audio feedback as an effective/ innovative aspect although only ~50% of the class had received audio feedback): 1 st year students: Before getting audio feedback I was certain that I wouldn't like it!However, for me, I find it more helpful listening to the comments rather than reading them.This is because the explanation of the feedback is clearer.The audio feedback is clear and easy to understand.Written feedback is occasionally difficult to read and understand, due to handwriting or the way the comments are written.It felt more personal and is more instantaneous because it doesn't have to be collected, it"s accessible as soon as it"s released and can be listened to anywhere.Written feedback allows me to jump to the exact point I am looking for rather than waiting for "lecturer" to speak.
Although audio feedback is useful, written feedback suits my learning style better and can be looked at in places when a computer is not available.I like both audio and written.Written is more permanent and you can pick it up and look at it any time and audio gives more detail so both cover everything.
Using audio for feedback on assessments: tutor and student experiences

Tutor Evaluation
In our experience opinion is divided amongst teachers in UKHE as to preferences for on-line versus hard-copy assessment of assignments.The provision of detailed handwritten or typed feedback on student assignments is undoubtedly time-consuming and the level of detail that can be provided is severely limited by the time required to write or type the feedback comments.With our experience in the provision of both audio and typed feedback, our findings suggest that the time taken to assess a word-processed laboratory report on-screen and provide feedback via audio is certainly not more than the equivalent process where feedback is provided via typed comments; indeed, if can be less by 10-20% although this is likely to be both tutor and assignment dependent.What is clear from the student feedback is that the feedback provided to students via audio is richer and more detailed than is possible to provide via handwritten/typed feedback within the same timeframe.It is simply not practical to write what can be conveyed so concisely via the spoken word, where the desire to elaborate upon a particular point and/or to cite illustrative examples is not constrained to the same degree by time considerations.A disadvantage is its immediacy on returning to the feedback at a later date (as a student or tutor).Typed comments within a word document and their links to a particular section of the student"s work are simultaneously and immediately visible, which is not the case for audio, and this has been cited by some students as a drawback of audio feedback.

Conclusions
The use of digital audio for feedback on assessments is straightforward to implement and provides a low cost alternative to written feedback on both hard-copy and electronic assignments.Student feedback on audio suggests it is richer, more detailed and more comprehensible in comparison with written feedback (handwritten/typed), although some students indicate they would still prefer to receive written feedback and highlight some drawbacks associated with accessibility and navigational issues with audio.We are now working to design curricula that will enable our students to develop the skills required to engage routinely with a variety of feedback in a systematic and meaningful way 9 , forming part of the increased focus on development of graduate attributes/employability skills.As part of these developments we have started to accommodate, as far as practical and on a limited scale, students" feedback preferences and it will be interesting to see how prominently audio feedback features as and when it becomes more commonplace and whether audio-based feedback modes are demonstrably more effective than written feedback.It is clear that there is considerable scope to extend the use of emerging digital audio and audio-visual technologies to diverse areas of teaching and learning in UKHE.

Figure 3 :
Figure 3: Extract from an assessed laboratory report that accompanies audio feedback via an mp3 file st time just looking out for score. 2 nd time listening to the feedback.To fully look into areas of improvement needed.Once without lab diary, then again with.To go through the feedback again.
It was more detailed than having notes in the lab book.Better than illegible handwriting.Specific to me, better feedback than a poorly thought out written assignment as with most other subjects.This combined with annotated feedback on lab report was excellent to really identify specific areas to improve.Worked very well if you had your lab book with you.More in-depth than written notes.It was less formal and stressful than a face-to-face interview.I also enjoyed the ability to pause (I could then take notes).I could also go back and listen again for reference in other lab books in other modules.Explained notes in book in more detail.Better than face-to-face because you can listen again.Was good going through the feedback while going through lab diary.Feedback more detailed.ForQ6, (what did you dislike about this audio feedback?)although only a few students provided a written answer, the issues mentioned most frequently by this group of students were the lack of opportunity for contemporaneous dialogue and navigational issues (see below).Only one student said audio feedback was not as effective as written feedback.
liked the fact that it was very specific to my own work and the fact that I could listen to it whilst scanning through my lab book to see areas in which I could improve.
I personally prefer written feedback as I find it easier to refer back to; however with the assessment of a lab diary I feel that audio feedback is particularly suitable.The marker would otherwise have to either write all over the lab diary or virtually write an essay in order to deliver the kind of detail that was given.I preferred written feedback as I felt fully informed as to what I generally needed to improve on and what I did well in.With the audio feedback, I got easily bored and therefore didn't feel motivated to listen to it all, whereas for the written feedback I could easily scan through what was said and refer back to it whenever I needed to.It is not as easy to refer back to when completing future work for example; you would probably need to listen to the whole thing again to find a particular point.As it is audio, we always have to replay it to look over it and if we only need a specific part to look at then we will have to find the exact part from the audio.I prefer written feedback on my work as it is easy to look over it and it"s easier to understand the mistakes if it is written on the same page.I don"t mind audio feedback either, but the only disadvantage I found was if we need a specific part from the lab diary, we will have to find the exact part from the audio.