Student at Risk : Identification and Remedial Action System for Improving Retention on Computer Science Programmes

Retention has been a serious problem for computing programmes in the Computer Science and Creative Technology Department (CSCT) at the University of the West of England (UWE): for example in 2013-14 the BSc (Hons) Computer Science programme lost 18% of its year-one intake. Addressing this, CSCT developed a three-pronged strategy comprising retention research, the monitoring of module performance and the development of a student-at-risk-identification-and-remedialaction-system (SRIRAS). The last initiative was prioritised in 2015-16. An intern team was recruited to run a system to monitor student attendance and academic performance across seven CSCT programmes, identify those students most at risk and then, with the collaboration of programme leaders and yeartutors, help them. Help included phone calls, email warnings, and face-to-face meetings. The latter led to problem identification, advice on non-academic problems, encouragement to attend more lectures, tutorials and peer assisted learning (PAL) sessions, encouragement to engage with module staff, setting up special PAL sessions, and advice to attend catch-up programming sessions. Results indicated that many of those most at risk are struggling with non-academic concerns such as starting the programme late due to a visa problem, or not being able to attend due to a grant not arriving. Results also indicate that some students seem to be being “saved” while others do not. But overall, they indicate that the introduction of the retention system was mixed.


Introduction The context and problem
In her 2014 report on undergraduate retention and attainment across the disciplines, Woodfield (2014) reports on disciplinary differences in higher education for the academic year 2010-11.She found that Computer Science was the discipline with the lowest continuation rate (91%).A student was deemed to have continued if they had either progressed into the next year, graduated on their initially registered award (or a higher award) or were in a "dormant or writing up" period (typical of Open University students).Computer Science made up 4.2% (67,847) of the student body in 2010-11.
At UWE, Bristol, this extremely low national continuation rate for the Computer Science discipline has also been observed for the ten computing programmes in the Computer Science and Creative Technology (CSCT) Department, including for the BSc (Hons) Computer Science programme itself.In 2010-11, 22% of students failed to continue.For 2012-13, a faculty-wide system was introduced whereby a team of two interns contacted students identified by the programme leader to be at risk.Continuation improved for Computer Science students: 11% failed to continue.However, this system was not made available

Introduction
The two key roles for running the SRIRAS are those of the Retention Coordinator (RC) and the Programme Leader (PL).Throughout the 2015-16 academic year, the RC performed the following two key activities:  Acquired student engagement and academic performance data  Maintained a database of students at risk During the same period, each PL periodically performed the following two key activities:  Reviewed their list of students at risk  Took remedial action for each such student

Retention coordinator's role
The RC acquired the following data for each student on each of the ten CSCT programmes:  Attendance data  VLE (Blackboard) engagement data  Formative and summative assessment results Attendance data was acquired for all core module lecture, laboratory and tutorial sessions.This data was considered a proxy for engagement.Another proxy was taken to be the log-on record of each student on the VLE.
Academic performance data comprised students' results for formative and summative assessments.

Programme leader's role
Periodically during 2015-16, each PL, optionally along with the RC and the programme's year-one Academic Personal Tutor (APT), reviewed the list of students at risk.For each student, the team would decide whether the student was at risk, or at critical risk.Students at risk were judged to be those who had some academic performance failures or attendance failures.Students at critical risk were judged to be those who were failing to attend nearly everything and performing poorly academically in everything.

Remedial activities
Following the deliberation and decision making process, the PL and, optionally, their RC and their year-one APT, implemented the decisions in order to help the students.
Students at risk were sent emails pointing out why they were deemed to be at risk and encouraging them to engage more fully and to take steps to perform better academically.
Students at critical risk were invited to attend a face to face meeting with their PL or their yearone APT.At each such meeting an attempt was made to ascertain the cause of the student being at risk.Advice was then offered dependent upon the cause.Students with very serious problems, for example health or financial problems, were advised to speak to the Student Support Advisor, full time professionals; others were advised in a variety of ways including, for example, advice to start attending lecture, tutorial and laboratory sessions, advice to attend lunchtime programming surgeries, advice to seek help from module staff and advice to attend Peer Assisted Learning (PAL) group sessions or individual PAL sessions.
The next section presents the main outcomes of running the system in 2015-16.

Outcomes
At the end of the academic year 2015-16, the impact of the SRIRAS on academic performance was assessed across the ten CSCT programmes.The results of the assessment are shown in table 1 in the following subsection.Table 2 presents data on the academic performance of students who have been at critical risk on the Computer Science programme during the year; it also presents data on the causes of them being at risk.

Retention rates across programmes
Based upon June 2015 and June 2016 Award Board data, Table 1 presents the predicted losses of students from the ten CSCT programmes for both 2014-15 (columns two and three) and 2015-16 (columns four and five).The sixth column shows whether retention has improved for a programme (positive change in percentage) or declined.
These results are discussed in the next main section.

Causes for students being at critical risk on the Computer Science programme.
For just the Computer Science programme, table 2 presents the academic performance of each student on the programme identified as critically at risk during the year.Academic performance is measured in terms of the number of core modules failed.
The table also summarises for each student the main reason why they were at risk.This data was gleaned either from email replies or from face-to-face interviews.
This table is discussed in the next main section.

Discussion
Retention rates Table 3 shows the average continuation rate for both all programmes in UWE and for all programmes in the CSCT Department over five recent years.For each year, the continuation rate is worse for CSCT programmes than for programmes in general in UWE.And from table 1 it can be seen that retention is still an issue in 2015-16 for computing programmes at UWE: for example, five out of ten programmes are predicted to lose 20% or more of their cohort.It can also be seen that four of the programmes have improved their retention rates.For example, Broadcast Audio and Music Technology is predicted to go from 71% to 100% in 2015-16 than in 2014-15.
Nevertheless six programmes have a worse retention rate in 2015-16 than in 2014-15, which was not the hoped for or expected result.
One reason for this unexpected result may be that PLs varied in the extent to which they enacted the SRIRAS.The results obtained so far for a questionnaire issued to the 10 CSCT PLs indicate that this may be the case: while at least one PL enacted the SRIRAS fully from induction week to the end of semester two, another only came on board towards the end of semester one.Other PLs seem to have fallen between these two stools.

Causes
Examining table 2, it can be seen that there is a range of reasons why students become critically at risk and perform poorly academically.The reasons do not seem to be related to their academic ability or willingness to study; they seem rather to be related to nonacademic causes such as, for example, becoming mentally ill, having to spend time responding to family problems, or realising that either Computer Science or even university itself was not the path for them.

Successes
It is worth noting here the two clear SRIRAS successes.S12 and S13 were two students who were critically at risk for most of the year, one because he missed the first four weeks of semester one, the other because he was ill during induction and the first week of the semester.Both had failed to absorb some of the key information necessary for becoming viable students.However, following intervention and remedial action by both the programme leader and the year-one APT, S12 passed all four year-one core modules and S13 passed three and is predicted to pass the resit.

Could more have been saved?
Reflecting on the range of reasons that seemed to be the cause the poor academic performance of critically at risk students, it seems that little more could have been done to obtain better academic performance.However there were three students whose reasons are not yet known, as well as some others from the list who may have performed academically better and thus have been saved if more of the causes of poor retention in computing courses (Gordon, 2016) had been better addressed.For example, if social bonds between computer science students had been stronger, then five students as well as some of the three "Not known yet" may have performed better academically.

System limitations
This section presents the main problems that were encountered in running the system and suggests ways of addressing some of them.

Monitoring student attendance
In the first few weeks of the first semester, the monitoring of student attendance by module leaders and tutors was reasonably good.However as the semester progressed, the enthusiasm of some members of staff waned, as they themselves reported.
So a reliable method is needed for accurately monitoring attendance at all teaching sessions for the whole year.The university is investigating SEaTS (SEaTS), an IT system that will do this, and there are other IT systems which could also be investigated for this purpose.

Face-to-face interviews
Students at critical risk who were invited for a face to face interview by the year-one APT or PL often accepted the invitation, but then did not attend it.However, the year one APT was very persistent, and in semester one, he eventually saw all of the students at critical risk.However in semester two, even replies to such invitations dried up.So it was often difficult to assess how students were progressing with the advice given to them in semester one.And it is not clear how this problem can be resolved.

PAL sessions
In addition to suggesting to students at critical risk that they should attend Peer Assisted Learning (PAL) sessions, special PAL sessions were also set up for the at risk students.However, quite often these students did not attend these sessions, which was disappointing.It is difficult to see what more could be done to encourage them to attend.

Early
warning of poor academic performance During the year it was discovered that most official, summative assessments came towards the end, or at the end, of semester two; there were very few in semester one.This meant that the best early warning of poor academic performance came from unofficial formative tests.However, because of their unofficial nature, not all of the students attended such tests; so important, potentially useful diagnostic information was missing for some students who later, at the end of semester two, showed up as critically at risk.Department management staff have been made aware of this problem.They have also been encouraged to ask core module leaders to consider introducing a summative assignment in semester one.This might be a research-based assignment to take account of insufficient material having been covered early in semester one to undertake a conventional assignment.
In addition, module staff will be made aware that a portfolio-based assignment might be introduced.This would have a final submission in semester two, but with earlier and strict phased hand-ins of its constituent parts in both semester one and semester two.Either solution would be likely to produce early academic performance data for all of a programme cohorts students.

Programme leaders' performance
The variable nature of the quality of the enactment of the SRIRAS by the PLs, referred to earlier, is being addressed by advice on, for example, how best to use the year-one APTs and the need for face to face meetings in order to help students at critical risk more.It will also be addressed by similar information being imparted to them at the Departmental Away Day, which takes place at the beginning of the academic year.

A
Students-at-Risk-Identification-and Remedial-Action-System was operated for the ten computing programmes in the CSCT department at the University of the West of England in 2015-16.Mixed results were obtained: retention improved for some programmes and some students were saved; however, retention declined for other programmes.In addition, the results indicate that causes for poor retention are many and varied; and they are not all based in poorly motivated or incompetent students.

Plan for 2016-17
In 2016-17, it is planned to run the SRIRAS again.However, more attention will be paid to ensuring that all PLs run the system well.In addition, some assignment due dates have been brought forward in the academic year, allowing students' academic performance to be assessed earlier than in 2015-16.
A retention framework will also be developed.This will identify actions that may be taken to encourage retention over the whole of a student's "lifetime" with the Department, from reading the prospectus before applying through to graduating three or four years later.Many of the actions will be intended to support a sense of belonging in the students, a feature defined as critical by Thomas (2012) and also by Jordan ( 2016)

IT tool support for retention
The University is investigating the SEaTS retention software.The SRIRAS project staff will be monitoring this initiative.

Table 1
Retention rates across CSCT programmes

Table 2
Causes for students being at risk on the Computer Science programme

Table 3
Average continuation rates for the CSCT Department and for the University