An evaluation of a STEM summer undergraduate research internship scheme : student-perceived learning gains

The organisation of a STEM-wide summer undergraduate research internship scheme to ensure equal opportunities for all students is reported. Student engagement in the internship programme is much greater in the Biological, Pharmaceutical and Chemical Sciences (142 applications in 2016) than in Engineering and Computing (30 applications in 2016). To evaluate the impact of the scheme, previous interns were surveyed about their perceived learning gains from doing an internship. Twenty three responses were received from 46 previous interns. The greatest gains were reported for key skills such as: project and time management and working independently. Though the number of responses was small, engineering interns reported lower perceived learning gains than science interns. The survey indicated that perceived gains in key skills such as oral presentation and scientific writing were not rated very highly. In addition, more oral presentation and writing a final report were rated amongst the lowest for changes that might improve the learning gain from doing an internship.


Introduction
Summer Undergraduate Research (SUR) experiences are available in most UK universities.These typically involve a penultimate year student being paid to work on a research project with a member of academic staff for 6-10 weeks.However, the number of opportunities available to students depends on the institution and the discipline area.University-wide schemes are present at a number of UK universities e.g.Warwick, Kings College, Cardiff, Sheffield, Sussex and Reading.In these schemes SUR experiences are available to students from all disciplines.The list of funded projects between 2011-2015 at Sheffield University, as part of their SURE scheme (Sheffield University , 2016), reveals good uptake across all STEM disciplines, as well as in arts and human science subjects.
Elsewhere in the UK, SUR experiences are organised and funded at departmental level, for example, for Biological sciences York, Birkbeck, Dundee, Leeds, and Newcastle run departmental schemes.Across the UK schemes are also run in Maths, Medicine, Physics, Engineering, Psychology and Sociology departments.Universities also promote national undergraduate research bursary schemes funded by charities, research bodies, and professional societies e.g.Wellcome, Royal Society of Chemistry, Royal Academy of Engineering and EPRSC.In these cases students apply to the funding body with details of a proposed project and supervisor (RSC, 2016;Wellcome 2016).They may also allow students to study at an institution different from their own.In a review of research careers in the UK, Thrift (2008) argued for research councils to develop a national research experiences programme for undergraduates.Some action has been taken on this recommendation  the EPRSC (2016) run a Vacation Bursary Scheme (VBS)  but these are available only to Universities with EPSRC Doctoral Training Partnerships (DTPs).In this scheme a university can award up to 15 bursaries per year.
Healy and Jenkins (2009) (1998), where students' learning can be extended by working closely with a group of experts in their discipline.
In the United States of America, SUR internships are long established and there is a significant body of research evidencing the gains that students get from such experiences.Seymour et al. (2004) reported that in interviews with 76 students, five mains types of benefits were identified by the students: personal/professional gains (28%); development as a scientist (28%); skills development (19%); clarification of career plans (12%) and preparation for future study/career (9%).Hunter et al. 2007 subsequently developed the Undergraduate Research Student Self-Assessment (URSSA), an extensive survey containing multiple questions exploring each of these gains.This has since been used to further evaluate such schemes (Thiry et al. 2012;Stanford et al. 2015).Thiry et al. (2012) reported that good supervision and authentic tasks that were part of a community of practice were important in providing a high quality experience.Lopatto (2004Lopatto ( , 2007) ) developed a shorter Survey of Undergraduate Research Experience (SURE) in which interns were asked to measure their learning gains for 20 items.The largest gains reported by students were for "understanding of the research process", "readiness for more demanding research", "understanding how scientists work on solving problems", "learning laboratory techniques", "tolerance for obstacles" and "working independently".Interestingly the bottom four gains were: clarification of a career path, skill in oral presentation, skill in science writing and learning ethical conduct.John and Creighton (2011) adapted the URSSA to measure the gains for UK SUR interns.The conclusions of this work were that interns developed a better understanding of the research process and confidence in engaging in research but that they are unlikely to result in large increases in postgraduate recruitment.They also identify that a suitably challenging project is important to these gains.Surveys by Russell et al (2004) in the US indicated that undergraduate research opportunities actually clarified students' interest in research and encouraged students who had not thought of postgraduate study to pursue it.Research also shows that work experience in the form of placements or internship helps success in gaining graduate level employment (Bennett et al., 2008;Pennington, Mosley and Sinclair, 2013;Moore, Sanders and Higham, 2013).Overall the evidence suggests that SUR experiences can help clarify future research and career plans but are unlikely to change the plans of many students.Russel et al. (2004) reported that in a survey of US STEM students, participation rates in undergraduate research opportunities varied between disciplines ranging from 34% in mathematics and 37% in computer sciences to 72% in chemistry.John and Creighton (2011) saw a similar variation in engagement with SUR internships with biological sciences and physical sciences (115 and 83 interns respectively) having much greater activity than engineering disciplines (25 interns).Craney et al. (2011) investigated perceptions of undergraduate research experiences across the sciences, humanities and social science, but not engineering.They found that the ratings of general benefits were consistent across disciplines; however some outcomes such as gaining technical skills and valuing the production of papers did vary across disciplines.Stanford et al. (2015) found very few statistically significant differences in the personal and professional gains made by STEM research interns compared to non-STEM interns.No comparisons between science and engineering within STE were reported.
In this paper the operation of the scheme in the Faculty of Science Engineering and Computing at Kingston University is described.Each year far greater engagement is seen from students and staff from biological and chemical science disciplines compared to those from Engineering.Previous evaluations have indicated that there are a number of reasons why students do not apply for SUR internships (Williams and Jones, 2016).
We were interested in exploring what students at Kingston gained from internships and whether there were any differences in the experience of interns from different disciplines.
Here we report the results of a survey of student-perceived gains of doing an internship in 2015; the survey was based on the SURE survey developed by Lopatto (2007).

Undergraduate Research Internship Scheme at Kingston
Over forty SUR internships within the Faculty of Science Engineering and Computing are funded each year.Each internship covers a stipend of £1200 for 8 weeks work and £300 for project consumables.
Academic staff are asked to submit short proposals for research projects that are realistic for undergraduate students working for eight weeks.A booklet of potential projects (100 in 2015) is circulated to all second year students and also third year students on fouryear courses.The internships were also advertised on the University student job site.A condition of doing an internship is that the interns must write a weekly blog on their experience of doing an internship.When advertising the internship scheme, potential interns are encouraged to read the blogs written by previous interns to get a feel for what an internship involves and to see the positive experiences of the vast majority of interns.To apply, students are asked to submit a cover letter and CV and list up to four choices in rank order.The selection of internship projects to be funded for each department in the faculty is dependent on number and quality of student applications.Around five students are shortlisted for each project and their applications are sent to the project supervisors.Supervisors then interview all shortlisted candidates and report their ranked choices.Each student gets feedback from the supervisor on their interview performance.Students can be interviewed for more than one project.The Faculty co-ordinator then allocates students to the internship positions based on student and staff ranked choices.
In 2016 there were 190 applicants and 45 internships were awarded (Figure 1).This was an increase on the number of application in 2015 (Figure 2).There is considerable variation between subjects area.In both years the schools of Pharmacy and Chemistry and Life Science had by far the greatest number of applications.
There is also varying engagement of staff from different schools as the number of projects offered differs considerably between departments and also between years.There is no obvious correlation between the number of project offered and the number of applications; for example in 2015, there were 21 mechanical engineering projects to choose from, but only 4 students applied, compared with Pharmacy and Chemistry where 11 projects were offered but 48 students applied.The number of applications in a subject area appears to depend on the type of projects being offered and also how well they are promoted by academics in the School.There is also evidence from the literature that internships are less common in engineering (John and Creighton, 2011).

Measuring student perceived learning gains from doing an internship
An analysis of the student intern blogs has previously revealed that students from different disciplines identified different gains from doing their internship (Williams and Jones, 2016).A number of studies have sought to measure learning gains from an undergraduate research experience.Kardash (2000), asked students to rate a range of their own research skills before and after doing a 12 hours a week undergraduate research experience for an academic year (32 weeks) or a summer internship of 40 hours a week for 8 weeks.Gains in a variety of research skills were reported.Others have designed and used learning gain surveys to evaluate the student perceived benefits of Undergraduate Research experiences.These include the extensive Undergraduate Research Student Self-Assessment (URSSA), Hunter et al (2007) and Thiry et al (2012) and modifications and Creighton, 2011) and the shorter Survey of Undergraduate Research Experience (SURE) (Lopatto 2007).

Data collection
To evaluate the scheme at Kingston a survey of 14 questions derived from SURE (Lopatto, 2007) was produced.The questions asked students to rate their perceived gain for a range of skills on a scale of 1 (no gain/not applicable) to 5 (very large gain), the same as the scales used by Lopatto (2007).

Skills surveyed
1. Ability to interpret results or information 2. Project skills 3. Time management skills 4. Planning experiments 5. Practical skills (laboratory or workshop technique 6. Working in small teams 7. Overcoming obstacles in a project 8. Ability to read and understand primary literature 9. Giving an oral presentation 10.Scientific writing 11.Keeping laboratory notebook 12. Self Confidence 13.Working independently 14.Knowing what you want to do after graduating i.e. career path The survey also asked students about aspects of an internship that might have increased their learning gains.They were asked to rate their level of agreement (strongly agree to strongly disagree) with the statement "I think my learning gain from doing a research internship would have been greater if there had been…"

Results
Twenty three responses were received from 45 interns.It is interesting to note that key skills such as working independently and project and time management skills had the highest rated gains; this is likely to be due to them being needed for all the internship projects.Practical skills and keeping a laboratory notebook were amongst the lowest.This can be attributed to some of the projects not being laboratory based; Figure 4 shows that these three skills have a number of responses giving a zero, whereas there are no zero responses for the project and time management skills and working independently.These results are fairly consistent with those of Hunter et al. (2012) who found students reported the most significant gains on doing an undergraduate research experience to be: confidence, understanding of how science research is done, the ability to work independently and post-graduation/career plans.The survey used in our evaluation was based on that developed by Lopatto (2007), who reported the largest student gains as "Understanding the research process", "readiness for more demanding research", "understanding problems are tackled" and "learning laboratory techniques".Lopatto found that "skills in science writing and oral presentation" and "clarification of career path" were reported as having the lowest student perceived learning gain.Again this was consistent with our findings.When asked to rate their agreement on whether changes would improve the learning gain, students were less likely to agree that the gains would be improved if there were more opportunities for oral presentation and a formal requirement for a final report (Figure 5).One aim of the project was to see if there were any differences between engineering and science interns' views of their gains.Unfortunately, this is rather difficult with only 7 responses from engineering interns.However, Figure 4 does give the general impression that engineering students rate gains in generic skills and research skills less highly than the science interns.This may be because they are doing the internships for different reasons and may also explain why internships are generally less prevalent in engineering disciplines than in science.

Conclusions
In line with many other studies the student reported learning from doing an internship are high, indicating that students value the experiences highly.No firm conclusions of differences between Science and Engineering interns can be drawn from the small population surveyed so far.The intention is to survey students in future years to build a bigger sample, as other have done (Stanford, 2015).Another issue to be addressed is that most studies, including ours, have focused on student-perceived gains.As Linn et al (2015) has highlighted self-reporting is not the most reliable method for measuring gains and further research into the impact of undergraduate research experiences on students learning and graduate outcomes is desirable.

Figure 1 Figure 2
Figure 1 Projects, application and awarded internships by department 2016

Figure 5 Figure 6
Figure 5Likert scale responses for changes that might increase learning gain from doing an internship