COMMUNITY DIRECTIONS Work-related learning: benefits and challenges

Universities are often criticised for not adequately preparing students for the workplace. In particular, the testing of taught content through examinations does not reflect the open nature of most roles, which require the application of knowledge. embeds employability development and applications throughout the course. In 2018/19 two work-related group assessments were introduced, in which first year students work collaboratively with companies and other students on real-world problems. This paper discusses the implementation and evaluation of the two projects, including the challenges and benefits, as observed by staff, students and clients.


Introduction
The teaching of Mathematics and Statistics at undergraduate level often focuses on the passive absorption of mathematical theory through lectures and the testing of procedures in exams which often does not effectively prepare students for the workplace (Cobb, 1992;GAISE, 2016). As presented by Tynjala et al (2003), "learning in a workplace environment is very different from …in a university environment". Hills et al (2003) investigate the gap between skills developed through degree programmes and those required in industry, and explore how workrelated learning can be used to address this difference. Educational literature suggests that active (Tishkovskaya & Lancaster, 2012;Freeman et al, 2014), collaborative (Roseth et al, 2008) or problem-based learning (Hmelo-Silver;2004;Marriot et al, 2009) increases understanding and the creation of more authentic project-based assessment techniques (Chance, 1997;Garfield, 1994;GAISE, 2016) allows students to apply their knowledge and develop their problem solving skills more effectively.
The Mathematics degree at Sheffield Hallam University sits on the applied end of the spectrum of Mathematics degrees. Based around applications, technology and employability skills, the programme already included numerous project-based assignments. However, despite many realistic examples, the students had little contact with external companies as part of their degree (unless they did a placement year in industry or a work-related final year project). This paper describes the introduction of two assessed work-related learning projects, where groups of students work directly with external companies. The projects, implemented within the first year of the degree programme, replaced existing assessments within two modules.
The work-related learning projects were motivated by the following aims: • Enable students to apply taught knowledge and see the relevance of learning material • Develop students' employability skills • Increase student engagement, motivation and confidence • Provide useful resources for local companies and showcase student abilities

The work-related learning projects
In their first semester, students create an Excel template for local businesses as part of a Peer Assisted Learning scheme. In the second semester, students apply their taught statistical knowledge to authentic research projects using data provided by one of a number of companies. In both projects, the students work closely with companies to discuss requirements, progress and results through meetings, written reports and presentations.

The Peer Assisted Learning (PAL) scheme
The PAL Scheme was introduced in 2008/09 (Waldock, 2010) with the aim of easing the transition into university. Groups of first year students carry out an assessment task whilst being supported by a pair of "PAL Leaders" who are students in later years of their studies. Further information about the scheme is presented within Waldock (2011) andCornock (2016). However, there have been several subsequent changes, including: permitting second year students to be "PAL Leaders"; the introduction of student "PAL Supervisors" and a timetabled "PAL Session" when all students are free.
The introduction of work-related learning in 2018/19 (as outlined in this paper), is the largest change to the PAL Scheme. The original project consisted of students investigating a mathematical topic of their choice. This was summarised in a poster, report and presentation. The new work-related learning assignment requires the students to create an Excel template and user guide, which are provided and presented to their clients at the end of the project for their own use. The projects in 2018/19 included: break-even analysis; investigating user demographics and creating data management templates. The student groups are allocated a project after providing a list of their preferences. The students meet their client early on in the project, and can contact them throughout.

Statistics assignment
Statistics is traditionally one of the most disliked courses within the curriculum and typically does not prepare maths students for the workplace (Cobb, 1992;GAISE, 2016). A number of changes have been made to the module to try to address this (Marshall, 2018) including the implementation of work-related teaching and assignments. Project based analysis and assessment was embedded within the module in 2017/18 and students particularly liked the final group presentation assessment which was fairly open but with a dataset which was familiar to them. The second work-related project, which replaced this assessment, enabled students to apply their taught statistical knowledge to authentic research projects posed by one of a number of companies, using data provided by the company. The motivation was that the clients obtain valuable statistical results whilst the students consolidate and develop their statistical understanding and thinking.

Methodology
At the end of the PAL Scheme in 2018/19, questionnaires were completed anonymously by: first year students, "PAL Leaders" and "PAL Supervisors". All the students were asked how they found the scheme in general and how it could be improved. They were also questioned about: their thoughts on doing a real project for a real client; the development of employability skills; putting their skills to use; the level of challenge and their confidence levels in the PAL Scheme. In addition, the "PAL Leaders" and "PAL Supervisors" were asked about their thoughts on the changes to the assessment task (as they had experienced a different task in their first year).
The first year students were also asked to fill in a questionnaire after the statistics project; unfortunately, as it was the end of term, very few responses were received. However, students on the Mathematics course are encouraged to regularly fill in reflective logs about all modules (which are monitored by staff) so these comments provide another source of feedback. Some of the comments are presented within this paper.
The clients involved in the PAL Scheme completed a questionnaire (which was not They were asked about their experiences of working with the first year students and reasons why they chose to offer projects. The clients for the statistics project provided verbal and written feedback.

Results
Almost half the first year students (45% of 76), 46% of the 22 "PAL Leaders" and one "PAL Supervisor" filled out questionnaires after the PAL project. Three of the five clients filled in the client feedback survey. Only seven students filled out a survey after the statistics project but comments were also collected through other feedback mechanisms (see Methodology). Unless otherwise stated, blank answers were excluded from the calculations and graphs.

Aim 1: Enable students to apply taught knowledge and see the relevance of learning material
When the first year students were asked for their thoughts regarding "doing a real project for a real client", 34% of the students made reference to the task being realistic or that their work would be used by an external company. They liked how they will be leaving university with experience because of this work and how the project gave them the opportunity to "practice for presenting for businesses in a work situation". A couple of students acknowledged the difference between the Excel work taught in class and the work they had to do in the project. They said they "had to use skills beyond what [they'd] already learned" and that they used Excel "in a different environment".
Students were asked how much they agreed with a number of statements on a five point Likert scale about working on the PAL project and the percentages agreeing with each statement are summarised in Figure 1.
Whilst most students acknowledged that the PAL project allowed them to see the relevance and application of parts of the course, it should be noted that the PAL project is not directly taught in a module (although Excel is formally taught in other modules). 100% of the students taking the survey after the statistics project agreed that the project allowed them to apply knowledge and see the relevance of statistics in the workplace.

Aim 2: Develop students' employability skills
All the student groups were asked which employability skills they developed or improved during the PAL project (they could tick as many as applied from a list of 11, with the option to say none of them). 97% of the first year students picked at least one skill, with an average of 6.3 skills. Figure 2 summarises the percentage of first years who selected each skill. All of the "PAL Leaders" selected at least one skill, but with a lower average of 4.3 skills.
The first year students and "PAL Leaders" also recognised that they developed skills during the PAL Scheme in previous years before the introduction of work-related learning (Cornock, 2016). However, it was found that working on a project for an external company meant the students experienced a richer learning experience as explored in the subsequent sections.
Aim 3: Increase student engagement, motivation and confidence Engagement was high amongst the first year students within the projects, with almost all students contributing to the work in the PAL and statistics projects and with more enthusiasm than traditional projects.
Several students mentioned how working on a real project for a real client motivated them to do their best as "it had to be useful". Another said "the work felt a little more important than other assignments, since [the company] might actually be using the template in the future". One student commented that they "felt like we were being exploited", but also responded positively about doing something real.
Many of the "PAL Leaders" were positive about the introduction of work-related learning into the PAL Scheme, with 70% preferring the new task over the one they did in their first year. They liked that the work would be used by a client, that the students took the work more seriously and that the project had more meaning. They thought the students "applied themselves to the task well and in a professional manner" and that the project "created a deeper understanding of the business world and a more personal connection to the project". The "PAL Supervisor" thought that "the clients will benefit from maths students providing mathematical advice and business strategy towards their business".
As the "PAL Supervisors" had been a first year student and a "PAL Leader" before the task was changed, they were also asked to describe any differences in the experience for the first year students between the two experiences. They said that the engagement of the first year students made it easier for the "PAL Leaders" and the new task "makes the PAL group work closer together and actually take a real productive stance to first year uni work".

Figure 3
Boxplots showing the spread of confidence scores for first year students before and after the PAL project Although some students found working with clients stressful at the start of the project and specifically said meeting the client was "quite intimidating", they generally liked how their work would be used. There were several comments specifically about working with a client. Students said doing a real project for a real client "felt like it was a big responsibility" and "added an aspect of severity to it, with real consequences".
The first year students were asked how confident they felt about undertaking an open project at the start and end of the PAL project on a scale of 0 (not confident) to 100 (very confident) and the spread of scores are shown in the boxplots in Figure 3. They were also asked to elaborate on their reasoning and, if their confidence levels had changed, whether such changes occurred at any particular time(s) during the project.
The average score before the project was 43 with 23% giving a score of at least 70. After the PAL project the average response of 66 with 55% of them being 70 or more. It was found that 74% provided a higher number at the end of the project than at the start. A paired t-test showed significant evidence (t=5.91, p< 0.001) to suggest a change in confidence with open projects with students changing by an average of 22.5 points [95% Confidence interval: (5, 30)]. Reasons for this change included: group work and support from others; communicating with clients and practising professional conversations. A number of students (26%) said that they realised the project was not as difficult as they thought it would be; they became more confident when they either started or completed some work, or the change happened when they received graded feedback. Students who said their confidence had decreased did not provide reasons.
When asked whether they noticed a change in confidence levels within their "PAL Group", all but one of the "PAL Leaders" (86%) indicated that they had. When asked what they noticed and when the change happened, the majority indicated that it was after the first few weeks, when students were settling into their friendship groups and the project. The results of Cornock (2016) indicated that the original PAL Scheme generally increased the confidence of the students (prior to the introduction of work-related learning). However, the findings of this paper suggest that working with external clients has further contributed to increased levels of confidence in 2018/19. There was a further benefit to the statistics work-related learning assignment. Statistics modules are generally disliked by mathematics students who often fail to appreciate their relevance, and this lack of engagement impacts on effective learning (Gal and Ginsburg, 1994). During the two years where the focus has been gradually changed to project-based statistics, attendance and module feedback has improved. The largest change in attitudes to statistics has been as a result of the work-related projects with several students saying that it was their favourite module. Whilst student understanding of the practical application of statistics was better after the changes in 2017/18, the improvement in understanding of statistics generally after the work-based assessment was more noticeable and this was reflected in improved exam grades.

Student comments included:
"At first I found stats boring and repetitive, however, as the year progressed I found it more interesting especially when completing the group project towards the end of the year, it highlighted how useful stats is in the work place and how it is used." "Really enthusiastic about stats now, even wish to seek a career in it." "On reflection, this was one of my favourite modules." Another benefit of the open statistics project with supervision was that students could work at their level, with weaker students reinforcing their understanding of basic techniques and the strongest groups learning new statistical techniques. Students were graded on communication skills and innovation alongside the statistical content and often the statistically weaker groups were able to shine in these areas, which were prioritised by the clients. One student who had struggled with previous statistics assessments, but loved working on the work-related project, commented "I looked at the project and thought, I can really do this and do it well". That student's confidence with statistics improved significantly through the project and she went on to get a first in the exam in addition to the work-related project.

Aim 4: Provide useful resources for local companies and showcase student abilities
There were three clients who provided feedback on the PAL Scheme -these were from a small business, a consultancy and a charity. Their reasons for choosing to offer projects to our students were as follows: "My company is very new and I needed any support/guidance/information available to me" "I wanted to see how a particular issue could be approached. For a relatively limited input it would give the students experience and potentially give me solutions I hadn't the time or knowledge to come up with" "As a small charity, any project work like the SHU Maths students completed for us provides valuable insights that we otherwise wouldn't be able to obtain due to limited staff and resources" The clients found the experience of working with the first year students positive and were very impressed with the standard of the work the students produced. They were surprised by how little the students needed support from them during the project, with one in particular mentioning that "the groups seemed to do all of their own troubleshooting". The clients liked how they could watch the student presentations at the end of the project. Further comments from the clients regarding the PAL projects are given below: "The work was excellent. Project X will be used as the basis for setting up tracking / recording my finances" "Two of the three groups in particular produced very user-friendly systems which had clear potential for us to input further data into in future, giving them further usability for months and even years to come" "All of the work was well thought out and will be of use to us as a charity" "All 3 groups worked really hard and produced outputs that we felt would be useful and valuable to the organisation"

Figure 4 Stages of a statistics work-related assessment
The companies taking part in the statistics projects were not asked about their reasons for taking part, but the responses after the projects suggest that they were not expecting to get as much from the projects as they did. The response from the external companies was overwhelmingly positive, particularly about: the benefits to them of working with the students; the usefulness of the work and the quality of their written reports and presentations. Most of the clients had worked with other departments but said they found our projects the most enjoyable and were very impressed with the standard of our first year students. One charity representative came to the last lecture to: personally thank the students working on her projects; reinforce how useful their statistics skills and knowledge are in many workplaces and encourage students to approach other charities who would benefit from their expertise. A second business invited one group to visit the company and has since offered them individual student placements. Since observing the benefits of using statistics in the workplace, the company is also keen to collaborate with academics on further research. Some of the comments from clients are given below:

Challenges
Whilst the benefits to students and overwhelmingly positive feedback from the clients were clear, the time invested by the staff involved was considerably more than with traditional assessment. Whilst both projects replaced similar existing assessments within two modules, the process of setting and Planning: Lecturers meet with Venture Matrix and possible employers sourced. Lecturers produce several flexible project outlines to meet client needs and requirements of the assignment. Generic marking schemes for different aspects produced Data: Messy data (missing values, outliers, recoding) or data unsuitable for addressing original questions Supervision: Regular meetings with lecturer encouraged Initial meeting: Students met clients after some initial data cleaning/analysis to ask informed questions Techniques: Students were required select at least two appropriate statistical tests from the course. Some learnt more advanced techniques. Analysis: SAS used to carry out selected techniques Communicate: A report and presentation suitable for a non-statistical audience and a 'work-log' for the lecturer to check statistical understanding were submitted marking the assessments required a number of changes and some adjustments to teaching material to account for the range of skills and knowledge needed for the different projects.
Although initially the swap from one specific dataset to data provided by different companies seemed straightforward for the statistics assignment, Figure 4 summarises the different parts of the planning and execution of the statistics assignment to emphasise the differences from a more traditional assessment.
Both work-related learning projects are going ahead in 2019/20 as the benefits outweighed the challenges. The largest challenge was the time spent by staff implementing the projects but the following points summarise changes to address this.
• Staff time spent on developing suitable projects and marking schemes will be significantly reduced in subsequent years and a larger time allowance for assessments has been allocated to the academics involved. • The team who help us find companies has example outlines and projects to share with clients so more suitable projects are sourced from the outset and at an earlier date. Example data is also requested before initial meetings for the statistics project to ensure the research questions can be addressed. • "PAL Leaders" next year will have undertaken the new projects and are therefore able to share their experiences and knowledge.
• Statistics peer support is being introduced to reduce supervision needed from lecturers. This includes support with SAS and writing reports. • During summer 2019, some of the students volunteered for extra work with the main statistics lecturer to improve resources for students undertaking open statistics projects. These include online SAS resources so students can quickly find the code they need and an interactive online test chooser which asks a series of questions to help the student choose the right technique and leads directly to paper resources. These new resources add to existing resources for SPSS and R which are available to students at any institution through the Sheffield Hallam University Mathematics and Statistics support pages: https://maths.shu.ac.uk/mathshelp/

Conclusions
In the work-related projects, the first year students developed employability skills, increased in confidence and deepened their understanding of taught material more than in traditional assignments. In particular, working with a real project for an external company made the work more realistic, motivated them to do well and allowed them to develop further employability skills that could not be gained through standard project assessment. In addition, the statistics assignment had a considerable impact on attitudes towards statistics, engagement and understanding. The external clients were very impressed with the students, the usefulness of their work, the quality of their work and some students have been offered placements as a result. Whilst feedback was overwhelmingly positive, the projects did initially require significant development from staff (although the workload will be substantially less in the subsequent iterations) and some students felt under pressure to perform better than in a standard assessment. Overall, embedding work-related learning within the curriculum is beneficial to students and companies, with students being better prepared for their future careers.