Closing the Feedback Loop : Collaborative Design of a Musculoskeletal Revision Course

Clinical Teaching Fellows (CTFs) at Leicester Medical School decided to supplement the learning needs of first year students on the musculoskeletal (MSK) module. Mid-unit evaluation had demonstrated that students had remarked positively on CTF teaching and many requested further anatomy teaching and demonstrations using prosections. It is increasingly important to provide students with evidence that their feedback is being acted upon, therefore CTFs collaborated in providing additional learning resources in the form of CTF-led revision courses while the MSK module was still ongoing. A survey was designed which aimed to engage students and to further explore their learning needs when developing the course. Based on these responses, two half-day CTF-led revision courses were designed, which included educational methods and topics the students themselves had suggested. CTFs collaboratively developed eight different stations, with one CTF designing and delivering the teaching material. Attendance was high and feedback indicated this was a valuable learning experience for students, with particularly positive responses about the interactive nature and high quality of the teaching. This experience demonstrates the benefits of working in partnership with students when developing learning activities, closing the feedback loop to improve student satisfaction, and collaborative planning when designing revision resources.


Introduction
Clinical Teaching Fellows (CTF) at Leicester University Medical School contribute to teaching of the Musculoskeletal System (MSK) unit for first year students.This unit has undergone many changes this year as a result of the new curriculum which increased the clinical content of the module ("Leicester Medical School Curriculum Redesign," 2016) Mid-module feedback indicated students had concerns about their learning of basic anatomy and their ability to clinically apply their understanding.Many students had asked for more anatomy teaching, including specific requests for CTF-led sessions and more time with prosections (a dissected part of a cadaver).
Student evaluation can be difficult to respond to "live" within a module, and this lack of action on feedback can lead to further student dissatisfaction.Interestingly, one of the new sections in the National Student Survey (NSS) for 2017 is "Student Voice" which contains the question "It is clear how students' feedback on the course has been acted on", acting as a measure of student engagement ("Review of information,"2016).Closing the feedback loop and making students aware of the impact of their feedback is becoming ever more vital to improving student satisfaction.The requests for more CTF-led sessions provided an opportunity to implement immediate action for current students in response to their feedback and to close the feedback loop.
When discussing this feedback, the CTFs reflected on the role of student engagement in the module.Feedback alone provides insight and student voice, but there was scope for actively engaging students further in the teaching and learning activities.Student consultation could provide students with choice and influence over design, and go some way towards involving students as partners in this process.Student partnership has many motivations and benefits, and in this case the role of student consultation was explored, as the first of the qualitative stages of student engagement in four-stage model of student engagement from the NUS/HEA student engagement toolkit (Healey et al 2014).

Methods
In response to these learning needs, the CTFs decided to run two half-day revision courses, outside of the module timetable; one half-day focussing on the lower limb and another half-day focussing on the upper limb.The aim was to develop a learning environment which would aid first year students in their revision of MSK anatomy and to respond to the mid-unit feedback.A further aim was to use feedback from this revision course to suggest positive changes to the new curriculum.
In order to foster student engagement, and to make the teaching and learning as high quality as possible, the revision course was designed with student partnership in mind.The learning needs and students' own ideas about effective teaching methods were explored via an electronic survey which was sent to the entire first-year cohort (table 1).
The results from the survey showed there was a high interest in the sessions, with 160 responders and over 90% indicating they would attend both days.37.8% requested anatomy teaching in the dissection room using prosections and models, followed by 32.7% requesting clinical case studies in small group work rooms.Of the 12.2% who requested "other", most specified wanting a combination of these two most popular options.The main topics requested were tallied and split into four upper limb and four lower limb sessions, which would explore one main MSK subject and incorporate related topics within.For example a lower limb session was "the lower leg", which further addressed other student-identified learning needs within it, for example clinical applications such as neuropathies of the region.
Each CTF chose a topic and designed their own materials for their session.There was a collaborative approach with respect to agreement that sessions would be interactive, multi-modal and incorporate use of prosections.Learning materials varied and included competitive quiz games, identifying anatomical features on prosections, interactive 'dance' routines and clinical conundrums.CTFs critiqued each other's materials and session plans and cooperated in ensuring there was minimal overlapping of content between the time-pressured sessions.The year group was divided in half and then into four smaller groups, who rotated in a round-robin fashion every twenty minutes.As attendance was voluntary, each group was an average of 10-15 students.
Two post-course evaluative online surveys were sent electronically to all students.The response rate was 67 for the lower limb course and 40 for the upper limb course.The surveys were qualitative, with free text boxes asking for comments about each individual teaching session, as well as general comments about the whole course for that day.

Results
The qualitative results from the post-course evaluative survey were initially tallied according to comment theme, classing them as either 'positive' or 'negative'(Figure 1).Within these two themes, student responses were coded by identifying specific aspects the students used to validate their overall positive or overall negative response (Figures 2 and 3).This process of thematic grouping of qualitative student feedback gave us the opportunity to represent and review the results in a quantitative manner.From the 107 total respondents to the two postcourse evaluative surveys, the coding of student free-text responses allowed us to extract 982 overall individual comments.As detailed in Figure 1, 82.3% of student comments were positive and these 808 comments are further represented on Figure 2. The largest group of positive comments (286) were non-specific in nature and gave a generalised positive response without using a key descriptor to explain their positive feedback.The top three positive comments which identified a specific aspect of the session included the method of delivery (17.1% of positive comments), the use of prosections (9.2% of positive comments) and the use of clinical correlates (7.8% of positive comments).Students also spoke positively about the organisation of the sessions, the use of technology and incorporating exam-style questioning into the sessions.Of the total comments extracted, 17.7% gave a negative overall response, the descriptors of which are detailed in Figure 3. Within the negative comments, the majority of students (29.3%) wanted extra time during the revision session.Whilst 10.9% of comments gave a generalised negative response, three aspects the majority of other negative comments focussed on included the wrong volume of information (13.8% of negative comments), the request for more exam-style questioning (9.2% of negative comments) and the better use of technology (8.0% of negative comments).Students also commented that they wanted smaller group sizes, more clinical correlates and more interactive activities.

Discussion
The aim of this teaching case study was to act on student feedback live and to explore the role of student partnership in design and development of the course, in order to actively engage students and enhance their learning experience.

Examples of Student Comments Theme
Thank you so much for all taking the time to teach us; you didn't need to and yet you did and we all really really appreciate it.

General Positive
The CTFs in this unit really have saved the day and are an invaluable resource to the med school and should be commended for it.

CTFs to lead more teaching
If we had more time this would be much better, 2 hours would have been good Wanted extra time Overall thought it was really well organised and structured very well, would highly recommend

General organisation of the session
Excellent use of repetition to aid memory and recall.Excellent use of mnemonics and alliteration to help us remember.Compartmentalised all the information very well so it wasn't overwhelming.

Use of mnemonics
Really appreciated the flag style anatomy that we will see in our exams as well.

Delivery
The game show style was really engaging which allowed me to actively learn.

Use of quizzing
Our results showed that generally the response from the students was positive, and that the majority of positive comments (35%) were non-specific generalised positive responses that focused on how much they had enjoyed the sessions, indicating good student satisfaction.However, an enjoyable student experience cannot guarantee increased acquisition of knowledge or improved exam performance, which opens up a potential area to study in the future.
Another strong theme in the positive feedback was that students responded well to the multi-modal teaching methods which were requested in the pre-session survey.138 positive comments were about the delivery (e.g.location in the dissection laboratory), 74 positive comments were about the use of prosections, and 63 positive comments were about the use of clinical correlates.
Demonstrating to students that we are responding to their feedback and suggestions by closing the feedback loop helps show the students that they have a voice that is listened to.As well is increasing student satisfaction, this has the potential improve student engagement with feedback in the future, as they will know it has the potential to change their own teaching experience, and not just that of future students.One potential downside to this approach of developing materials based on feedback could be that students may rely on tutors-led revision sessions rather than using initiative and developing skills to find or produce resources themselves.Following the revisions session, 16 students had specified that they would like CTFs to lead more teaching sessions in the future.CTFs are often junior doctors, many of whom have recently graduated from medical school, therefore have up-to-date experience of clinical application of medical knowledge.CTFs' roles now include some involvement in module development and, using feedback from this course, have suggested changes to the MSK module which include CTF-led tutorials within the module timetable.
The largest proportion of negative comments (43%) were made in relation to the balance of session length and amount of material covered in that time, with 29% stating that they would have liked longer sessions and 14% stating they would have preferred a different volume of content.In retrospect, we also agreed that longer sessions would have been valuable as they would have allowed us to cover more material and in greater depth.Unfortunately, the short 20 minute sessions were selected as a result of timetabling limitations and staff availability, so it is unlikely we could have increased their length on this occasion.In the future we may change the course format to include longer sessions; however there is a concern that if the sessions were elongated to allow incorporation of more material, there is the potential risk that students may become unengaged which would make the sessions less effective.
Another point raised in the negative feedback was that 24 students would have preferred a more multimodal approach, with 16 students specifically stating that they would have liked more quizzing/ exam-style questioning and 8 stating that they generally would have liked more interactivity.This is also supported by the fact that in the positive feedback, 50 students commented on how they had enjoyed the quiz elements, and 23 commented on enjoying the interactive activities.Interactive teaching activities have the benefits of increasing student engagement and supporting active learning.We plan to increase these elements in future sessions that we develop.
Moving forward, it is clear that working in partnership with students by acting live on their feedback and suggestions has been a largely positive experience, and that there are many avenues for future research, including exploring whether this method of developing teaching materials actually improves student learning or performance.Student partnership in the form of consultation about Interactive teaching activities have the benefits of increasing student engagement and supporting active learning.
development of new materials builds upon the passive student voice provided by end-of-module feedback.Consultation, in the form of the survey, actively encouraged students to make decisions about their learning, and we found that students were engaged more as a result.There are many more opportunities for richer and more radical student partnership which could be explored in future, such as active student-tutor collaboration where they having joint control over decisionmaking about course design.

Figure 1 :
Figure 1: Total Number of Negative Comments and Total Number of Positive Comments

Figure 3 :
Figure 3: Pie Chart Showing the Breakdown of the 174 Negative Comments

Table 2 :
Structure of the musculoskeletal revision courses

Table 1 :
Survey questions sent to first-year cohortWill you be attending the Lower limb session on Monday 20th March?

Table 3 :
Examples of Comments and Thematic Analysis