Investigating students understanding of organic reaction mechanisms from performing organic chemistry experiments

The use of laboratory work to improve students’ knowledge of theory is one that is disputed. Student reflections of what they learn during practical work repeatedly show that students rarely think about theory. There is a lack of data that objectively compares students’ knowledge of theory where they complete an associated experiment to when they do not in order to understand if practical work does effect students’ knowledge of scientific concepts. In this work we aimed to address this gap by investigating the effect of students’ knowledge of organic mechanisms where they both perform and associated experiment and where they did not. Our results showed that organic chemistry experiments had no effect on students’ knowledge of organic mechanisms. These results support the view that there is little evidence to support the use of laboratory work to aid understanding of theory.


Introduction
Laboratory courses play a pivotal role in undergraduate STEM degree programmes. They mainstay of such programmes, and despite the time and cost pressures of delivering laboratory courses, there is broad consensus that it provides a valuable learning environment.
In a traditional laboratory course, the primary purpose of performing experiments has been to improving students' understanding of theory. This is despite scant evidence that laboratory work is effective for this purpose (Kirschner & Meester, 1988;Domin, 1999;Hofstein & Lunetta, 2004;Reid & Shah, 2007).
Students rarely think about the underlying concepts that underpin experiments whilst they are in the laboratory (DeKorver & Towns, 2015;DeKorver & Towns, 2016;Galloway & Bretz, 2016;Domin, 2007). Instead, the overarching goal for students during an experiment is to finish on time. Students do learn how to perform practical techniques but rarely think about why they are doing them (DeKorver & Towns, 2015;DeKorver & Towns, 2016). At the start of university, students expect to use laboratory work to improve their understanding of theory (George-Williams et al., 2018), but these cognitive expectations decrease as they progress through the course (Galloway & Bretz, 2015b;Galloway & Bretz, 2015c;Galloway & Bretz, 2015d). Some students report learning nothing at all from performing practical work (Domin, 2007). However, some students do report learning theory when completing the post laboratory report (Domin, 2007). There is little evidence to support this claim.
Over recent decades there have been repeated calls to stop the use of laboratory work for teaching theory and instead use laboratory education to teach students how to 'do' science (Kirschner, 1992;Kirschner & Meester, 1988;Seery et al., 2019;Reid & Shah, 2007). The laboratory is an important learning environment for teaching practical skills, experimental design, problem-solving and professional skills. Laboratory education should focus on the development of these skills, rather than using practical work to enhance understanding of theory (Seery et al., 2019).
So why is the notion that laboratory work is necessary for students to understand theory persistently held? One observation of the aforementioned studies concerning engagement (or lack thereof) with theory during practical work is that they focus on students' own perceptions of what they have learnt. Findings are based upon analysis data of either surveys (Galloway & Bretz, 2015a;Galloway & Bretz, 2015b;Galloway & Bretz, 2015c;Galloway & Bretz, 2015d), interviews (Domin, 1999;DeKorver & Towns, 2015;DeKorver & Towns, 2016) and reflections (Galloway & Bretz, 2016).
What is noticeably lacking, from both sides of the argument, is objective evidence that either proves or disproves the hypothesis that students' knowledge of scientific theory is enhanced as a result of performing an experiment. In a rare example of such evidence, Turkish students completed an organic laboratory course, in conjunction to an organic chemistry lecture courses to aid their conceptual understanding of organic functional groups (Akkuzu & Uyulgan, 2016). Following these experiments, students completed worksheets and concept maps which were analysed. The results revealed no enhancement in the level of conceptual understanding for functional groups where students performed an experiment, and functional groups where students did not complete an associated experiment. Students were found to have a large number of misconceptions of functional groups regardless of whether they performed an associated experiment or not.
Clearly, more studies of this nature are required to establish if laboratory education should or shouldn't be used to improve students understanding of theory.
It should be noted that many of these studies have been from students performing expository-style (cookbook) experiments (Domin, 1999). In comparison to this style of experiment, Domin (2007) showed that students are more likely to engage in higherorder thinking with problem-based experiments.
Similarly inquiry-style experiments have been shown to improve students understanding of scientific concepts and engage students more in the theory associated with practical work (Duangpummet et al., 2019;Johnson et al., 2017).
Nevertheless, it is not uncommon for instructors to make the assertion that expository experiments are required for students to understand theory taught in an associated lecture course. Instructors holding onto this belief can also be reluctant to remove expository experiments in the redesign of laboratory curricula.

Aims of this study
The aim of this work was to determine if expository experiments improve students' scientific knowledge, in the context of organic chemistry. If performing an expository organic chemistry experiment did improve students' knowledge of organic mechanisms, then it can be hypothesised that their knowledge should be greater than for organic mechanisms where no associated experiment is conducted on students' understanding of organic mechanisms.
Herein, we describe our work to compare students' understanding of organic reaction mechanisms whereby students complete an associated experiment and where they do not. This work will help to establish whether expository organic chemistry experiments should be used to improve students understanding of theory.

Methodology
The University of Leeds is a researchintensive university in the UK. Chemistry students in the first year at the University of Leeds are taught fundamental organic chemistry mechanisms which include:

Mechanism (Associated Experiment)
Responses Mean (%) Standard Deviation  nucleophilic addition and substitution at the carbonyl group, nucleophilic substitution at a saturated carbon (S N 1 and S N 2), electrophilic addition of alkenes and electrophilic aromatic substitution.
As part of their first year laboratory module, students complete six organic chemistry experiments throughout the year. Five of these experiments are expository-style experiments, including a Grignard addition to a ketone, and a reduction of a ketone with sodium borohydride. For each experiment students are asked to complete a post laboratory report which includes asking for the mechanism of the reaction, and other questions to test understanding of theory.
An organic mechanisms quiz containing five mechanistic questions was designed for students to assess their understanding of organic mechanisms taught to them in their first year of study (table 1). Two of the five mechanisms, Grignard reaction and sodium borohydride reduction were based on experiments which students had performed in their first year laboratory course. The following three mechanisms, nucleophilic substitution at a saturated carbon (S N 1 and S N 2), electrophilic addition of an alkene and electrophilic aromatic substitution were all mechanisms which did not have an associated experiment.
The quiz was administered to students at the beginning of their second year course over a 3 year period. This was to ensure that students' were being tested on their mechanistic knowledge from their long term memory rather than from a recently revised exam or recently completed laboratory report. At the start of a second year lecture, each student was given one of the quiz questions at random to complete over a 10 minute period. This was to encourage the maximum number of responses from a cross-section of the cohort with minimal disruption to planned teaching. Answers were anonymous and had no bearing on students' grades. Students were informed about the purpose of the project and their rights to non-compliance. Ethical approval was granted by the host institution prior to this study.
Quiz sheets which were returned blank were disregarded. Mark schemes for each question were devised before grading responses. As the total number of marks for each question varied, marks were converted to a percentage to facilitate their comparison and will be reported in this manner alongside the standard deviation of each question. Independent t-tests were performed using SPSS to determine any significant differences between the mean scores.

Limitations
There are some limitations that should be considered when interpreting the data. Firstly, this study was conducted at a single institution. They may not be representative of students in other institution. Because each student only answered one question, these results may not be reflective of the entire cohort. No demographic data was taken. It could be possible that there could be differences in students based on demographics such as gender or ethnicity.

Results and discussion
The results for each mechanism question are presented in table 1. Independent t-tests verified the difference the Grignard mechanism score and the non-associated experiment mechanisms to be statistically significant ( < 0.05). Independent t-tests also verified that the difference between the sodium borohydride score was statistically significant to the S N 1 and S N 2 score ( < 0.05). However, the difference between the borohydride mechanism score was not significantly different from the scores for the electrophilic addition and electrophilic aromatic substitution mechanisms ( > 0.05). Independent t-tests also showed that the score of the S N 1 and S N 2 mechanism was significantly different from the electrophilic addition and electrophilic aromatic substitution mechanisms ( < 0.05).
If the expository-style Grignard and borohydride reduction experiments in the first year laboratory course enhanced theoretical understanding then the associated mechanism scores should have been significantly higher in comparison to the other three mechanism questions. Our results show that students did not enhance their understanding of Grignard and borohydride reduction mechanisms from performing these experiments in their laboratory course. In fact, students' knowledge of the Grignard mechanism was worse than the other mechanisms given in the quiz. Students' knowledge of the sodium borohydride mechanism was shown to be similar to their knowledge of electrophilic addition and electrophilic aromatic substitution mechanisms. The mechanism which students had the best knowledge, the S N 1 and S N 2 mechanism, was a reaction which students did not perform an associated experiment in their first-year laboratory course.
Cohen's d values were calculated to measure the effect of organic chemistry experiments on mechanistic knowledge by combining the results of the associated and non-associated mechanistic questions. (table 2). The measured effect size was deemed to be small to modest (Sawilowsky, 2009) in favour of the non-associated laboratory mechanisms. Even with a small effect, this is opposite to the expected effect if organic chemistry experiments improved students' understanding of mechanisms.
Overall, our analysis of student responses to both individual questions and combined lab and non-lab questions show that organic chemistry experiments have little effect on students' understanding of organic mechanisms. This supports our hypothesis that there is no correlation between students' understanding of organic mechanisms and their experience of performing an associated experiment.
These results also contradict the claims made by students that they learnt theory during post laboratory write up (Domin, 2007). The students involved in this study provide mechanisms in their post laboratory reports. If the claim made by students in the Domin (2007) study were true, then student performance for the Grignard and sodium borohydride mechanism should have been better.
The fact that students had the best knowledge of the S N 1 and S N 2 mechanism despite not performing any experiment involving this mechanism emphasises how insignificant laboratory work is for understanding of organic chemistry mechanisms. Instead, factors such as the quality of lecture-based instruction and use of active learning are far more likely to have a positive effect on student learning than laboratory work.

Implications for teaching
These results align with the growing body of research that there is no evidence that practical work aids theoretical understanding.
There are other valuable skills that students can learn from practical work and instructors should design their laboratory courses to meet those objectives.
Instead of designing laboratory curricula to align experiments with what is being taught in lectures, laboratory curricula should focus on the development of development of practical techniques, problem-solving skills, experimental design and professional skills (Seery et al., 2019). Seery (2019) proposes a framework where skills are developed in a step-wise approach, starting expository experiments to teach practical skills in the first year, then making the experiments more open-ended as the students move up each level to teach higher-order skills.

Conclusion
In this study we examined the knowledge of fundamental organic mechanisms of second year chemistry students to determine if performing an associated experiment had a positive effect on their mechanistic knowledge. Our study showed that student have no improved knowledge of organic chemistry when they complete an associated experiment in their laboratory course in comparison to when they do not. We recommend that instructors use laboratory work to teach skills where there is more clear evidence of learning gains than enhancing theory.