A qualitative analysis of responses to the web-based Calculus Conversation problems was conducted to examine how students addressed and attempted to solve the proposed problems.  This type of analysis is done when data are in a narrative form and allowed us to categorize responses into four groups, each representing a skill necessary to correctly analyze the problem. To ensure that the categories developed were clearly defined, comparisons were made between  two readers’ classification of the student responses.  Checking for inter-rater reliability is a key aspect of any analysis of text material. In this case, instances of agreement and disagreement were recorded with the total percent agreements shown in Appendix IV, Tables 11a,b,c.  The tables show that the percent agreements do vary, but all are over 50% with some agreements well over 80%. 

     Responses were categorized as:  (P or P-) focus of response was on technical or practical issues such as graphing, collection of data, or modeling and interpretation of data; (C or C-) focus of response was on conceptual issues of the problem through introduction of a new idea, proposed solution, or proposal of a question that spoke to key ideas; (I or I-) intercommunication where responses indicated a student had read and attempted to address an earlier response;  (L+ or L-) language and grammar issues that either made a response very clear or alternatively, incomprehensible; and, (Q) response posed a non-rhetorical question.  The minus (-) designation was used to indicate that the response confused rather than helped the conversation.    Responses could, and often did, receive more than one categorical index. 

     To illustrate the indexing of responses, in problem 3 the students were asked to analyze the toxic dumping activity of two companies and determine which, if either, was the better environmental citizen.  Completing the problem involved mathematically understanding the relationship between the rate at which toxins are dumped into a lake and the total amount of toxin that is accumulated in the lake as a result of the dumping.   Most of the responses to this problem (as well as problems 1 and 2) tended to focus on practical issues. 

A typical practical response: 

“Since we are dealing with GRAPHS, let me add something.. I think that the most logical way to starting is to define what the graphs will look like.  On the dependent axis (the bottom one), will be time.  On the independent axis (the left one) will be the TOTAL waste pumped into the river (is it a river?  I think so).  Because it is TOTAL, the graphs will never have a negative slope.  Whatever you do to lessen your output of waste, you are still going to have the amount of waste you had in the past…So the graph will not go down.  It is like the AIDS project.. That it was the total number of cases.. the best thing one could hope for was for the graph to level off.. so those are my two cents for now.” (JN) 

A typical conceptual response: 

“Although both companies reduced the amount of toxins they were dumping by 30%, the nuclear power plant increased the amount of toxins dumped before decreasing it, while Krusty’s only decreased the output of toxins. 

For example, say that at first they were both dumping at 20 gallons per day (I have no idea if this is realistic or not).  Then Krusty Burger worked for 12 months to decrease the amount of toxins dumped to 14 gallons per day (a 30% decrease).  The nuclear power plant, however, increased their dumping during that period, and although they to decreased to 14 gallons per day, if you looked at the total amount of toxins dumped theirs would be more than Krusty Burger’s.  Therefore Krusty Burger is doing a better job at helping the environment.” (BA) 

A portion of a response that indicates that a student read, understood and was attempting to reply to a particular student: 

 “I would like to mainly respond to Jeff’s comment that the slope of the Krusty Burger’s slope would be the same from t=0 to t =12.  It was stated in the problem that the pollution reduction rate was continually reduced until it reached 30%, but that does not mean that the rate of change was constant, and therefore the slopes at all times would not be the same. “ (SP) 

A response that poses questions for other students: 

“I think we need more info for this question.  For example, when the Nuclear Power Plant went off track, where were the two companies in terms with reaching their goal?  Were both of them already at 30% or were they some where in the middle?  When did the Krusty Burger reach 30%?  Did they end up reaching the goal at the same time?  I do not totally understand this.  If anyone else read something I missed please inform me.” (AP) 

     Listed in  tables 5a, 5b and 5c are the distribution of response categories given as a percent of the total number of responses within each section.  Evidence in these tables indicates that students clearly prefer practical approaches to problem solving.  This supports the claim by Brown, Collins and Duguid that students reason with laws, acting on symbols, looking for fixed meaning. [2] However, arriving at complete solutions to the problems required finding casual, transferable meaning in known laws and definitions.  The observed pattern for each problem showed that responses spiraled inward converging toward a correct solution. Early responses on the outer edge of the spiral were almost completely practical in nature.  Later responses were more likely to weave together the practical aspects of the solutions with their conceptual counterparts.  It is interesting to note that in all three sections on all three problems, the students never completely solved the problem on-line.  They hovered just slightly above the solution but were unable to land comfortably on a response that they could build consensus around.     Language extremes observed in problem 1 were almost non-existent by problems 2 and 3.   It is our conjecture that the students formed language norms in problem 1 that were used in subsequent problems.   Interaction among the students in the on-line conversation is clearly section- dependent.  The instructor in Section 1 provided a neutral introduction to the activity, made the announcement that a problem had been posted once, and subsequently gave every student full credit if they weighed-in. (Appendix III, Tables 10 a,b,c)  Instructors in Sections 2 and 3 provided an enthusiastic introduction to the activity, prompted students regularly to weigh-in early and often, and had a wider distribution of grades. (Appendix III, Tables 10 a,b,c)  It is not surprising that interaction among the students appears to depend heavily on both explicit and implicit messages given by the instructor. 

TABLE 5a
Problem 1

TABLE 5b
Problem 2

TABLE 5c
Problem 3