This page contains histograms of student responses on all of the questions that were deemed to relate to the question of what makes a good problem.
Question 1: Solving a mathematical problem usually involves finding a rule or formula that applies.
The three mathematicians responses were 1,1,3.
Looking at this histogram, we see a small change towards the responses of the mathematicians.
Question 10: A good mathematics problem often has many different solutions.
Mathematician Responses: 5,6,6
On this question, we see a more dramatic shift. Almost all responses that were antithetical to the mathematicians views changed.
Question 26: There is only one correct answer to any math problem.
Mathematician Responses: 1,1,1
Little change is shown in this set of responses.
Question 28: A good mathematics problem requires you to pull in concepts that have been previously studied.
Mathematician Responses: 4,6,6
In this question, we again see a shift in the direction of most of the mathematicians, most students were already of that mindset, but the shift shows a strengthening of the conviction.
Question 35: Most good mathematics problems can be done in a single sitting.
Mathematician Responses: 1,1,1
I am mystified by the student response to this. On the bright side, half of the students that agreed with this statements to start with did not agree at the end. However, three students moved towards more agreement (one of the 2s moved to a 1, while two of the 1s moved to 2s, and the third 1 changed to a 3. On the other hand, if one looks at distributions, the first distribution is bimodal, while the second appears to be closer to a normal distribution.
Question 36: Some mathematics problems have more than one correct answer.
Mathematician Responses: 4,6,6
This data shows a very nice change towards strong responses in agreement with the statement, similar to the mathematician views.
Question 38: A good mathematics problem can be represented in different ways.
Mathematician Responses: 5,6,6
Again, a fairly positive shift to strengthening mathematical attitudes.
Question 39: There are obvious ways to solve a good mathematics problem.
Mathematician Responses: 1,1,2
Again, here we mostly have a move in the positive direction, but it is not as clear cut as the previous question. (One student gave a before response of 3.5, which I have counted with the 3s). Two students moved from a 1 to a 2.
Question 41: A good mathematics problem brings together different areas of mathematics.
Mathematician Responses: 5,6,6
The responses for this question appear to move somewhat negatively.
Overall, students seem to move towards the mathematicians views on good problems. The strongest shifts appear to be on questions 1, 10, and 36. Question 1 refers to the depth of the solution (the solution usually requires finding a formula or rule), Questions 10 and 36 have to do with the existence of multiple solutions or methods of solutions. There appears to be a minor shift towards the mathematician responses on Questions 28, 35, and 38, all of which refer to how they are solved. With the exception of Question 41, the other questions show little or no change. Thus, it appears that for the most part, student attitudes towards what makes a good mathematical question moved towards the more mathematical.