Open Research Online Bridging the study gap: Provision of support for mathematics students during breaks in study

Many STEM subjects are strongly hierarchical: learning and progression depends upon threshold concepts and retained knowledge from previous study. To help progress learning, support can be offered during regular study periods, but what happens during breaks between studies? The requirement to provide continuity during study breaks is recognised as vital in assisting students to bridge the knowledge gap and have the best chance of success. At the Open University, we have found that over 64% of students studying one of our second level mathematics modules completed the prerequisite module over a year before starting the module. During this gap, their use of techniques required to study effectively may have become “rusty”. Even students who progress immediately find there are study holes where some of the retained knowledge has been lost. We describe the creation of a ‘Revise and Refresh’ support program that helped our students revise and refresh the knowledge required to study our level two module successfully. their and fulfil their potential. Through pedagogic content and


Introduction and background
At the Open University our mission is to be open to people, places, methods and ideas. We promote educational opportunity and social justice by providing high-quality university education to all who wish to realise their ambitions and fulfil their potential. Through academic research, pedagogic innovation and collaborative partnership we seek to be a world leader in the design, content and delivery of supported open learning. To quote Martin Bean, one of our former Vice Chancellors "There is no typical OU student. People of all ages and backgrounds study with us, for all sorts of reasons -to update their skills, get a qualification, boost their career, change direction, prove themselves, keep mentally active. The OU is open to them all." Since the 1990s Universities have acknowledged "the mathematics problem": the phrase used to describe students arriving at university without the mathematical knowledge required for their course (LMS 1995). Further to this they realised that students may have a problem studying mathematics and were not well supported. In response to this shortfall in support and in addition to their standard tuition, they began to offer 'learning centres', in which students may drop-in with questions relating to their studies (Perkins, Lawson & Croft 2012). These centres have been seen as a vital mechanism for helping students enhance their mathematical and statistical skills (Grove et al 2017). It was recognised that unless Universities provided appropriate forms of

Number of years since finishing the prerequisite module Percentage of students
Less than 1 year 20.13% 1 year 24.60% 2 to 5 years 26.32% 6 to 16 years 6.64% Did not study the prerequisite 22.31% learning support for mathematics and statistics then there would be an adverse effect on student satisfaction, retention and progression. Furthermore, Mathematics support is now more visible and high-profile within HEIs and is seen as important for enhancing the student experience and aiding success (Tolley & Mackenzie 2015).
Open University students register onto a degree pathway; once on a pathway, students register for the required modules on an individual basis. This means that students can blend study with other commitments and work through the degree at their own pace. Whilst studying a module, a student has access to many of the facilities of a standard face-toface drop-in centre as described by Matthews et al (2012). They are assigned to a tutor who can give help and advice as required; they also have access to forums where they can discuss problems with other students studying that module, and a timetable of study which gives a suggestion of when to complete each topic to prepare for the final exam. When a student is taking a study break they do not have an assigned academic tutor and contact with other students is on a more ad-hoc basis.
Given the geographical diversity of our students, hosting a face-to-face learning centre to serve students between modules would be impractical; the idea of a drop-in facility, however, remains relevant. Our response to this requirement is to provide a suite of distance learning support sites that help to prepare students for their next module; the sites provide support from tutors, together with dedicated forums in which students communicate with other students. In this paper we will examine the support centred on a specific second level module, MST224: Mathematical Methods.
By analysing the prior study of MST224 students ( Figure 1) we have established that over 79% of students that study MST224 either did not study the prerequisite mathematics module or have taken a break of at least a year since finishing it.
During that time their use of the techniques required to effectively study MST224 may have become "rusty", even students who progress to MST224 in less than a year will find that there is a study gap where some of the retained knowledge has been lost. Looking at the intake of students over 4 years of study from 2013 to 2016 in Figure 2 we see that the proportion of successful students decrease as length of time since taking the prerequisite module increases.
As part of our support for students preparing to study MST224 we wanted to create an active community of learners (Tinto 1975) and provide opportunities for student-to-student and student-to-academic staff interactions. Our structured, interactive bridging material should help to retain and support students during the transition between MST224 and its prerequisite modules, enabling them to be better prepared for the module, increasing their academic attachment to other students and to the University. It was designed to help to bridge the study gap and allow students to "hit the ground running" at the start of the module having already addressed prerequisite questions and met other students. Outline of the support offered: revise and refresh program Following the techniques described in Designing for student retention (van Ameijde, Weller & Cross 1016) and Designing online courses to promote student retention (Dietz-Uhler, Fisher & Han 2007), we designed a programme that supported the development of study habits and provided a gradual knowledge and skills gradient, helping students develop a growth mind-set (Dweck 2010). Participants will have an opportunity to develop and/or practice key IT skills that would help them progress with their degree pathway, such as inputting mathematical notation into interactive quizzes (used in our continuous assessment), and using online tutorials and forums.
The program is split into two key areas: • A revise and refresh website, which is student self-led. It contains an environment in which students can explore the areas of the prerequisite module that they need to review and receive guidance from tutors. • A series of tutor-led bootcamps run just before the start of the module; an academic-led series of online revision tutorials.
We will refer to timing within this paper, so we feel it helpful to detail the key parts of the timing in this outline: • July 2017: initial contact to point students to the resources; A central part to the project is the use of tutors both to harvest material for inclusion within the website, and also to facilitate the learning of the students. Tutors are our subject matter experts, and one of the single most valuable resources to which we have access at The Open University.

Revise and refresh website
Students are directed to the revise and refresh for MST224 website via a series of emails, starting in July, sent out at key dates during the run up to the module start. The website is also linked from the student's study homepage and can be accessed by any student registered with the Open University via our School website.
In this part of the program, we are emphasising the student's own ability to conduct independent learning. The students will need to be honest with themselves about their own ability; they will need to reflect upon their skills, and seek guidance and support to fill in any gaps. Students take ownership, accountability and responsibility for their own success in this part.
We employed three tutors with expertise across both the prerequisite module and MST224 to harvest existing examples, exercises and videos, who know the specific skills needed for success in MST224, and furthermore know the areas in which the students tend to struggle the most.
The website provides mathematical support targeted on six key areas vital for success in MST224: The materials were created with a graduation from "straightforward" to "more challenging" exercises; we aimed to engage the students during the leading months from July to September ahead of the October start for MST224. We used existing materials so that students could revise examples and exercises that they had seen before, and refresh their knowledge using material with which they were already familiar. This minimised the production workload, as the material had already been through significant editing and quality assurance processes.
The front page of the website has the following structure: • The first link directs students to a selfdiagnostic quiz; this assists the students in assessing their knowledge and ability to start MST224. Engagement in such quizzes has been shown to be an important indicator of student success (Calvert & Hilliam 2016). These quizzes exist for all mathematics modules at the Open University and help students to selfassess their abilities when considering the decision to study a module. Students are recommended to attempt the quiz before enrolling on a module, and it can be an important first step in ensuring they are prepared even after enrolment.
• A table containing the six topic headings with a suggestion on how long it takes to review each topic (measured in weeks) and the date of the bootcamp. The table is used as the portal for accessing the topics. • Finally, a link to the tutor support forum which is also linked from every subpage. At all stages the students are encouraged to post messages and questions on the tutor moderated forum. At key points throughout the summer the tutors post extension questions and practical situations that relate to the mathematics studied, inviting the students to comment on the posts, increasing their participation. The forum moderators and students also have the use of a virtual classroom if they wished to discuss any issues.
Each of the six topic areas are presented using the same structure (see screenshot in Figure 3). At the top of the page, we suggest that students complete a brief online quiz to establish if they need to spend time revising that area. Under that, the main topic is broken down into several subheadings which links to revision areas, a couple of sets downloadable of questions are available for more practice, a link back to the initial quiz so the students can see if they have improved after working on the topic and then a link to the website forum in case they have any specific questions.
Each online quiz consists of 12 questions: 10 of these are based on a range of subjects within the main topic which increase in difficulty and 2 questions are from previous topics for further revision. A student can attempt the quiz as often as they like and will usually get different questions each time. They are allowed three attempts at each question; after the first incorrect attempt they are given a hint of which section in the prerequisite module the question is testing (see Figure 4).  Well done, you've got all these questions on differentiation correct. You're probably ready to move on to refresh yourself on the next topic: integration.

70%-99%
Well done, you've got most of these questions on differentiation correct. You might wish to review some of the material and try some of the practice questions on this topic and then have another ago at this quiz to try and improve your score, but you are probably ready to move on to refresh yourself on the next topic: integration.
Don't forget that you can return to this quiz as often as you like to keep practising your skills.

50%-69%
You've make a good attempt at this quiz, and have got lots of question correct. However we would suggest you do some more work revising the topics related to the questions you got wrong and then have another go at the quiz to see if you can improve your score.

0%-49%
Your score indicates that you need to do a bit more work revising your skills on differentiation. Do look at the resources on the Differentiation page and give this quiz another go later to see if you can improve your score.

Figure 5
Overall feedback to computer-marked quiz With a second incorrect answer, the student receives a hint on how their answer was incorrect, or otherwise on how they might answer the question. A third incorrect answer gives the solution in full, providing again the reference to the prerequisite module and an option to try another similar question.
Inputting the correct answer will give the student a fully-worked solution, a reference to where the work is found in the prerequisite module and the option to complete another similar question.
Once the student has completed the full quiz they are given feedback overall on how well they have done ( Figure 5).
If, after completing the online quiz, the student feels that they need to review any part of this topic, they may return to the topic page and visit one of the subtopics. Each subtopic gives an overview of the subject, which is enough for a brief refresher and then a reference to the material in the prerequisite module if the student wishes to study it in more depth, together with useful videos.
Once the student has revised and refreshed all the subtopics, they can download a series of questions with full solutions to attempt in a traditional 'pencil and paper' style; they can then link back to the online quiz to see if they can improve on their score. Links to the tutormoderated forum are provided throughout.

Bootcamps
In the September before the start of MST224, a series of six bootcamps (online tutorials) are given in the virtual classroom associated with the revise and refresh website; each one concentrates on one of the six key topics (see page 4). They are run by two of the tutors that collated the resource material for the website; one tutor presents the bootcamp and the other tutor is there to assist by answering any questions that arise in the chat box and taking over presenting if the initial tutor has any technological issues.
The bootcamps are designed to challenge the learner, using effective questioning to achieve a deeper understanding. They are run in a relaxed and friendly manner, promoting discussion, with students encouraged to post as many questions as they wish into the chat box with time at the start and end of each session given over to more general chat creating a community of practice (Wenger-Trayner & Wenger-Trayner 2015). Each session is live with tutors actively solving questions in real time, which can help to remove some of the student's "fear of failure" (Swan 2005) as they see tutors and other Figure 6 Student interaction with the computer-marked quizzes, by topic and date students making mistakes and can welcome errors as a learning opportunity rather than something to avoid. A senior academic and member of the MST224 module team also attends some of the bootcamps to answer any procedural questions that arise and offer pastoral care and qualification guidance in a private breakout room.

Student Usage
There are two different ways a student may have used the revise and refresh website and bootcamps. They can either have actively used them by doing any of the following: • answering an online quiz; • posting or reading a message on the forum; • attending a bootcamp.
Alternatively students may have passively used the site by: • viewing the website; • downloading practice questions; • watching a recording of a bootcamp.
Unfortunately we only have methods of recording the participation of the active users, as analysed below. It is hoped that there may have been many more passive users that have also benefited from the resources within this revise and refresh program.

Online quizzes
In October 2017, at the start of MST224 there were 455 students registered to study the module. Of these students, 142 tried at least one of the online quizzes, with 43 students attempting at least 4 quizzes and 20 students attempting all 6. The revise and refresh website opened on 21 st July 2017 and all students registered onto MST224 at that time were informed by email.
In general the progression of the students through the online quizzes ( Figure 6) roughly follows the suggested timetable for studying the revise and refresh website, however there is a large influx of students doing the quizzes on all topics during the first few weeks in September; this coincides with the time at which most students register on MST224 and the series of bootcamps.

Figure 8 Forum interaction
A small number of quizzes are attempted even after the module had started on the 7 th October. The revise and refresh website are accessible to all students registered with the Open University and it is interesting to note that 45 students also attempted some of the quizzes even though they did not go on to study MST224 that year.
The online quiz covering Algebra had 121 participants, and was the most popular quiz (see Figure 7) this may be because it was the first, but also it is a topic with which many of our students struggle. The number of students participating in the quizzes reduces in line with the list of topics; this is as expected, as we may anticipate student's motivation and perseverance to drop as they progress through the more difficult topics. However more students took the differentiation quiz (5 th in the list of topics) than both the vectors & matrices and integration quizzes which could suggest another area of concern.

Forum posts
The students were encouraged to engage with the forums to ask questions of their support tutors, and to answer 'challenge' questions from the tutors. Figure 8 summarises the forum activity; notice the higher engagement of those students who Read forum posts, compared to those who engaged by posting.

Bootcamps
A reminder communication was sent to students registered on the module at the start of September about the revise and refresh website, and to invite them to attend the bootcamps. In total 62 students attended at least one bootcamp, 43 of these students had also completed some of the online quizzes, with 19 students only attending the bootcamps and not actively participating with the revise and refresh website. As with the online quizzes, participation in the bootcamps decreased as the series progressed with only 20 students attending the final bootcamp on integration (topic 6, see page 4).
We recorded that 34 students attended over half of the bootcamps, and 10 students attending all 6. These figures may seem quite low with respect to the number of students that are registered onto the module; however they are consistent with the data we collect on attendance at our module tutorials. Furthermore, these attendance records do not account for the number of students that will watch the recording after the event. Data on those that watch the recording could not be captured from the system used in 2017, however it should be possible to do so in 2018.
We recorded 99 students that participated in the online quizzes but didn't attend any bootcamps therefore in total 161 students, 35% of the total body registered participated in the revise and refresh program for MST224.

Retention and progression
When analysing retention we felt that as well as looking at students that had or had not participated in revise and refresh, it was also important to quantify their participation. We treated the participation in each of the six quizzes as a separate event; attendance at one of the six boot camps was also a separate event.
We have created a scale detailing student's engagement with the revise and refresh program for MST224 ranging between 0 and 12; on this scale, 12 represents that a student that participated in every available event, and 0 represents that a student participated in no events.
We have separated the students into several groups: • group 1: those that have had no engagement; • group 2: those that have had low engagement (1-3 events); • group 3: those that have had moderate engagement (4-8 events); • group 4: those that have had high engagement (9-12 events).
In Figure 9, we compare the marks that students gained in their first written (formative) assignment of the module on a per-group basis. We see that a higher percentage of students who had moderate and high engagement with the revise and refresh program gained higher marks in the assignment than those students who had less engagement or no engagement. Of course, we are not implying causation, but we might speculate that engagement with revise and refresh is one of the types of good practice that students can adopt in order to maximise their chances of success. The percentage of students that did not receive a mark for this assignment is highest for those students that had not engaged with revise and refresh, suggesting that they had also not engaged with the module now that it had started.
We may further look at students' behaviour at the start of the module. 18 students dropped out of MST224 in the first two weeks; of these students, 9 of them had participated in revise and refresh, with 3 having moderate engagement and 6 having low engagement. In our retention figures these students are deemed not to have started the module as they have stopped too early to have engaged fully in MST224. For the 9 students that did engage in revise and refresh, this may have been a factor in the reason they changed their mind on studying the module.
A further 28 students dropped out of the module before December 2017; of these students, only 6 had participated in the revise and refresh program before MST224 and no students were in the high engagement group. Therefore 79% of students that have dropped out by Christmas didn't actively participate in revise and refresh before MST224. Looking at this as a percentage of those that participated in events, this means that 8% of students who had not participated dropped out compared with 4% of those that did.

Figure 12
Percentage of students submitting assignments One of the reasons for creating the revise and refresh program was the low pass rates of students that had not studied the prerequisite module in over a year. Looking at the results from 2017, alongside the results from prior years ( Figure 10) we can see that generally this year, regardless of engagement with the revise and refresh program, all students have done better than in previous years. However, students that had engaged with revise and refresh in groups 3 and 4 had a significantly higher proportion of students passing than those in groups 1 and 2 (little or no engagement). Also it would appear that if students had not studied the prerequisite module the percentage of students passing was similar whether they had engaged with the revise and refresh material or not. For a student not to study the prerequisite module they must have gained equivalent experience outside of the Open University.
Referencing Figure 11, we see that students who engaged with the revise and refresh program in groups 3 and 4 had a lower proportion of Withdrawn or failed than those in groups 1 and 2. Students in groups 3 and 4 had the highest proportion of gaining a distinction.

Conclusions and extensions
Within the School of Mathematics and Statistics at the Open University we are moving towards a more continuous method of support where our students will have access to appropriate material and advice from an academic during all of the time they are considered students with us. The revise and refresh program is part of a suite of support, such as the early start programme (Calvert 2018) and module choice forums, run from our School Website which can be accessed by all registered students. Supporting students outside of regular study is nationally recognised as imperative for enhancing the student experience and retaining students as we move to give continual qualification support rather than module support which is restricted to the normal study program.
The revise and refresh program uses a two layer approach, the website is student led, they must take ownership and accountability for their own progression through the material, it can be studied at their own pace and in their own time. This is complimented by the bootcamp series which is a tutor led; the tutor ensures that all material required is covered and provides a real time opportunity to ask questions about the material. The two parts of the program are interlinked with the revise and refresh website raising issues that can be addressed in the bootcamps and the bootcamps exposing areas where the student needs to look at the material on the revise and refresh website in more detail.

Figure 13 percentage of students passing MST224
For 2017 we have had the largest percentage of students sitting the exam (Figure 12) since the module began and also pass rates ( Figure  13) have increased by over 3% on the previous year and are more than 1% higher than the previous highest pass rate.
For 2018 the revise and refresh program for MST224 has been extended to cover two other modules that have the same prerequisites. Several further revise and refresh programs are being prepared to cover our other level 2 modules and our level 3 modules and from 2019 onwards, via the support offered on the school website, students will have access to a support program for all compulsory modules in our BSc in Mathematics.