Improving Problem Solving with Simple Interventions

Although problem solving is a major goal for most science educators, many still rely on the demonstration method as an approach to teach it. This remains the case even though most are not happy with the results. Using a web-based problem delivery system to track students’ performance, we have investigated the effects of collaborative learning, and concept mapping on student problem solving ability. We find that student ability in general can be improved by about 10% after a group problem solving intervention. Furthermore we find differences in improvement depending upon the students’ level of logical thinking and gender.


Introduction
The improvement of problem solving abilities is a major goal of science educators 1 , and a great deal of effort has gone into finding ways to improve these skills.Unfortunately and despite a growing body of research on how people learn and develop problem solving skills 2 , many instructors rely upon the tried and true (or at least the traditional) method of demonstrating how the problem is solved and assigning similar problems for homework.Cognitive scientists tell us that knowledge is constructed by students and that skills must be developed by actively learning them rather than by watching another person's demonstration 3 .Yet faculty are slow to change their teaching approaches.Many faculty still give lectures about how to solve problems and then expect students to become expert problem solvers with no further assistance, even though a great deal of evidence (including students' test scores) indicates that traditional teaching methods do not result in optimal improvements in problem solving for many students 4 .
When asked about the performance of their students, most faculty will say they are dissatisfied.Why then do they not implement some of the newer pedagogies that have been shown to be effective in the classroom?A number of possible explanations arise.Perhaps it is because faculty are unaware that there is a better way -much of the research has been reported in unfamiliar venues.Perhaps it is because some scientists are unconvinced by research that often relies on qualitative observations.Perhaps it is because they prefer to blame unprepared students.Certainly some of the fault goes to the reward system in higher education that is not geared to excellence and scholarship in teaching 5 .Whatever the reasons, it is incumbent upon faculty to use the best tools available to do the job.We need to approach teaching and learning in the same ways as scientific research, rather than relying on the status quo, opinion, and hearsay.
While there are many excellent proven approaches for incorporating research based scientific teaching methods into the curriculum 6 , many of them require an investment in time and energy to use effectively.For a majority of faculty this change in teaching style may be difficult to accomplish given the time constraints and current reward systems.However, if we can incorporate relatively small changes into lecture based courses and observe real improvements, then the shift to more inquiry-based and active learning may gain momentum.One possible approach is collaborative learning, a widely used technique that can be employed in a variety of educational settings 7 and for which there is substantial research evidence to attest to its effectiveness 8 .For example: Mazur's use of Concept Tests 9 has shown measurable improvement in student understanding.Yet many faculty still do not introduce the relatively easily incorporated collaborative learning techniques into their classes.
Another simple intervention that can be used is concept mapping 21 .It has been reported that concept maps can provide students with a visual representation of their understanding of a given concept which in turn can promote metacognition 22 and motivate students to take the initiative to fill in gaps in their understanding.Research using concept maps has identified a relationship between problem solving and While there are many excellent proven approaches for incorporating research based scientific teaching methods into the curriculum, many of them require an investment in time and energy to use effectively Communication conceptual understanding.For example, Francisco and Nakhleh 23 reported the relationship between the quality of concept maps constructed and the performance on traditional chemistry problems.
In this paper we synthesise some of our previous research 24,25 on the effect of using a collaborative learning intervention on student problem solving abilities, and report a comparison of this with concept mapping as an intervention.Both methods are short and easy to implement -and result in measurable improvements in problem solving abilities.

Experimental Methods
We have previously developed and reported [10][11][12] on the IMMEX (Interactive MultiMedia Exercises) system that allows us to deliver case-based problems to students and to track the sequence of actions that they use to solve the problem.The problems used in this study involve scenarios in which students must identify an unknown compound by choosing which tests to run and which data to use in the identification.One problem, Hazmat, requires students to identify the unknown; the other problem, Lewis Structures, used in the concept mapping study also requires students to identify the Lewis structure of the unknown.For each problem there are multiple unknowns and each requires a different sequence of tests and inferences from those tests.The unknowns are not all of the same difficulty: it is more difficult to identify nitric acid than sodium chloride.Therefore we cannot use percent correct as a measure of student ability.Instead we use Item Response Theory (IRT) which takes into account the difficulty of the problem as well as the probability that the student has arrived at the correct solution 13 ; it is this student ability measure (which in our work ranges from 20-80) that we use in this report.

Research Design
Each study involved over 700 students who were enrolled in a general chemistry course at a southeastern research university.They were told of their rights as Human Subjects and completed informed-consent forms to allow their anonymous performance data to be analysed.All enrolled students were required to complete the assignment for course credit regardless of whether they gave permission for their data to be used.
Study1: Hazmat 24,25 The goal of the study was to investigate whether allowing students to work in a collaborative group would improve problem solving abilities.Students were required to complete at least five problems individually, followed by five or more problems in a collaborative group, and then five or more individually.This pretest, intervention, post-test experimental design was employed because it would allow us to compare the performance of students before and after the collaborative intervention.
We previously found 14 that students tend to stabilise on a problem solving strategy after performing fewer than five problems, and will continue with that strategy regardless of whether they are successful.We saw the same pattern in this study; that is, student abilities rapidly increased after the first problem attempt, and subsequently stabilised (Figure 1).Since these problems are quite complex it may take a student one or two attempts to learn to navigate the problem space and find the appropriate tests and information to identify the unknown.This finding is consistent with theories of skill acquisition 15 .Figure 1 shows that after the first three problem attempts the average student ability levels off (there is no significant difference between the abilities for performances 4 onwards).Previous studies have shown that the strategies adopted during this time are persistent and will be re-employed up to 3 months later 16 .
After the initial group of problems were solved by students individually, the students were paired up and asked to perform at least five more problems.Finally students worked individually on at least five additional problems.The whole experiment extended over the course of several weeks.Figure I shows the abilities of the pre-grouping individual performances as compared to the group performances and the post-grouping individual performances.
As presented in Figure 1, when students work in groups the average ability rises rapidly and levels out after three performances, and this improvement stays with the student after grouping.Note that the final set of data for post -grouping student abilities are fairly constant, and all the postgrouping performances have a significantly higher ability (p < 001) than the fifth pre-grouping individual performance.It appears from these data that allowing students to collaborate while solving problems improves their ability, and that improvement is retained after the students return to individual Improving Problem Solving with Simple Interventions

Issue 3
Communication problem solving.This finding is a direct rebuttal to those reluctant to allow collaborative learning in their classes because they feel the stronger students will dominate at the expense of weaker students.In this study we see that on average the improvement is about 6-7 units or about 10%.A question remains, however, about whether students of different intellectual abilities (as discussed in the following section) are equally affected by this intervention.
Students who participated in this study were also asked to complete the GALT (group assessment of logical thinking) test 17 which probes student understanding of proportional reasoning, data inferences and control of variables; all these skills are important in a science course.On the basis of their scores on this test, students were assigned to one of three categories of logical thinking based on Piaget's theories of intellectual development.
Formal: students are able to do proportional reasoning, make inferences from data, control variables and understand conservation of matter.Pre-formal: students who are pre-formal may be able to perform at a formal level on some tasks and not on others.Concrete: students' thinking levels are not fully developed; for example a concrete student is not able to reason from data, and may not be able to undertake many of the problem solving activities found in a college general chemistry course.
Previous reports 18 indicate that despite Piaget's original findings of formal thinking levels being attained by some as early as 11-14 years old, up to 50% of college freshmen students have not reached a fully formal thinking stage.In our study we found that 54% of the general chemistry students were Formal (F), 38% pre-Formal (P), and 8% Concrete (C).
Students were paired in all possible combinations ( F-F, F-P, F-C, P-P, P-C, C-C) and asked to perform the same problem solving sequence as described previously, (at least five individual, five group, five individual).When individual student ability pre-grouping is compared to student ability postgrouping, a number of interesting trends emerge as shown in Figure 2.
For most students the average gain is around six units (or 10% since the ability scale ranges from 20 The other noteworthy result is the gain in ability for pre-formal students who are paired with concrete level students, this being the only gain that is significantly larger than the average.A possible explanation for this finding is that, preformal students in these groups, are forced into the role of decision maker and teacher when paired with a concrete student.Our data provides evidence that pre-formal students can move into a higher thinking level.In fact, pre-formal students in a PC group have a final ability level of 56.5 which is identical to the final ability level of the formal students in any group.In contrast, the final improvement in ability for formal students following collaborative efforts does not appear to depend on the type of group in which they worked.
Furthermore, if these data are analysed by gender we see that most of the gain for pre-formal students emanates from the pre-formal female students as shown in Figure 3.
As can be seen, female students who are classified as preformal display marked improvements in problem solving ability after working with a group, although female concrete students do not seem to benefit in the same way.

Study 2: Lewis Structures
The goal of this study was to compare the effectiveness of collaborative grouping and concept mapping as interventions for problem solving.The study involved students in 45 laboratory sections with the labs being equally divided (15 for each designation) among concept mapping, collaborative, or no intervention (control).All students were asked to complete two Lewis structure problems.A week after completing them, the three groups of students were assigned in the laboratory either a concept map which was to be completed individually,

Communication
Improving Problem Solving with Simple Interventions or a collaborative Lewis structure computer assignment containing two problems, or another assignment involving an unrelated problem.The collaborative group composition was heterogeneous and random.Students were given at most an hour to complete these assignments.After completing the inlab task, each group was asked to complete four additional Lewis structure problems for homework.
Comparison of the abilities between the pre and the postintervention assignment did reveal subsequent gains in student abilities for both the concept map and collaborative interventions.Gains were also observed with the control group, but this was expected considering students' prior exposure to this problem.The concept map group had the highest overall abilities following the interventions while the control group had the lowest.
If the gains in student ability for males (Figure 4) and females (Figure 5) are viewed separately, we see an even more interesting trend emerge.It appears that different interventions are more effective depending on the sex of the student.For males, analysis indicated the post concept map abilities were statistically higher than either the collaborative (p<0.01) or the control (p<0.01)groups.That is, for males drawing a concept map was a more effective intervention than working in a collaborative group.For females, the opposite was true.The collaborative lead to higher gains than the concept mapping intervention for females.
The observed gender effects may be attributed to the visual/ spatial or verbal components of the intervention.Halpern 26 noted knowledge can be stored either visualspatially or verbally.Concept maps might promote visual storage by allowing one to connect the relationships among concepts and ideas in a diagram, while collaborative groups are more likely to promote verbal storage from the conversations and interactions that occur within groups.

Conclusions
Asking students to reflect on their thinking, either by discussion with others or by developing visual representations, while engaged in problem solving activities leads to improvements for most students, and these improvements are retained after grouping.That considered, the question remains: why do these methods have such a positive effect on problem solving, and why does this effect linger in subsequent performances?
An explanation surely lies in the fact that students are forced to become more thoughtful about their actions.That is group problem solving and concept mapping promotes metacognition 19 .Students must explain to their peers or themselves why they think an action should be taken and what the result might mean for their particular problem.The most significant outcome of this research is that students retain their improvements and are better problem solvers when working independently after a simple intervention.The inference is clear: even informal collaborative groups, and short activities in which metacognition is encouraged, are a valuable tool in the teacher's arsenal.They can lead to measurable improvements in student problem solving ability in a relatively short time, and they can be easily implemented.

Figure 1 :
Figure 1: A comparison of student abilities for pre-collaborative, collaborative, and postcollaborative performances.

Figure 3 :
Figure 3: Gains in student ability after grouping, by sex and group type.

Figure 4 :Figure 5 :
Figure 4: A comparison of the results observed for males.The 'd' values indicated above the bars indicate the gain in student ability between the pre and post assignments.The concept map intervention had the highest overall gain.

Gain in Individual Student Ability After Intervention by Group Type and Logical Thinking Ability
Gain in student ability for each type of group and the thinking level of the students in the group.The gain is statistically significant for every group at the p<0.001 level, except for the C-C group.
Problem Solving with Simple Interventions