This portfolio is part of the UNL Peer Review of Teaching program.

Return to Example Portfolios

Bernstein Project Revised Assessment Tools

Topic: Use of ordinary activities as motivators in an organizational setting

Reading: A journal article in which quality of employee work in a restaurant was improved by an arrangement in which time scheduled on the most preferred work activities was used as a reward for increases in measured quality of performance on least preferred work activities.

Old: What were the reinforcing consequences in the Welsh, Bernstein & Luthans [restaurant] study? How were the consequences identified? New: Suppose you were asked to implement a motivational program in a business setting. It is a small production unit with 25 employees who can engage in the following activities: production planning, inventory delivery, direct production, packaging, and marketing. Based on your understanding of the study of the fast food restaurant, how would you proceed to improve the quality of the employees' work by using access to activities as a motivator. Your answer should include the assessment of the relative values of activities, the establishment of contingencies, and a simple design for evaluating the project. Finally, describe the costs and benefits of the program for the business and make a recommendation about whether or not it should be implemented.

Variant: Suppose you were asked to implement a motivational program in a volleyball team. It is a competitive college team that practices three hours a day, including the following activities: passing drills, serving drills, hitting drills, videotape analysis, conditioning, scrimmage, and tactical play preparation. Based on your understanding of the study of the fast food restaurant, how would you proceed to improve the quality of the athletes' play by using access to activities as a motivator? Your answer should include the assessment of the relative values of activities, the establishment of contingencies, and a simple design for evaluating the project. Finally, describe the costs and benefits of the program for the coaches and make a recommendation about whether or not it should be implemented.

Comments: Given how similar the variants on each topic are, it is amazing to me that anyone gets under 100% on the second version of each exam. Year in and year out, however, the range of scores on retake exams continues to include levels of achievement from barely passing to nearly perfect.

My strategy for teaching these generalized skills began with conventional readings (some textbook material and some reprinted journal articles). Students prepare answers to study questions provided for the readings, and class time is spent having students offer their answers and respond to comments from me and from other students. One example similar to the exam questions is offered in class, with students giving answers and I provide feedback. Students take the first exam and we go over it in the following class, including explaining the grading rubric and answering questions about answers. Those who wish to improve their grade may take a second version of the exam which includes the variants on the original question themes.

Student learning scores in the fall semester support the notion that the new items were in fact more demanding; perhaps we can interpret that as evidence of requiring a deeper understanding. The previous two years in this same course an average of 45% of students scored 90% achievement or better on the first exam; this fall only 16% of learners achieved at that level on the upgraded first exam. The previous two years an average of 22% of students scored 69% achievement or worse on the first exam; this fall 33% of learners achieved that badly on the upgraded first exam. Normally about 20% of the class do D or F work on the first exam of the term, but I count on publicizing the high rate of excellent achievement by many students and offering the retake option to keep everyone going.

Some students gain so much on the retake exam (scores typically increase 10 to 40 percentage points) that they prepare less well for the first version of the second exam, counting on improving with the retake. Scores on the second exam last fall repeated the pattern of the first exam. The previous two years 35% of students scored 90% achievement or better on the initial offering of the second exam; this fall only 18% of learners achieved at that level on the second exam. The previous two years 35% of students scored below 69% on the second exam; this fall 55% of learners were below 69% on the second exam. While there was substantial improvement with retakes, the final achievement scores for the fall were lower than the previous two years; the grades were accordingly lower as well, as I made no adjustment in the scale. My plan is to make adjustments in how I teach, raising grades by improving learning.

During the fall offering I was very explicit with the class throughout the semester about how I was changing the measurement and how my goal was generalized use of conceptual tools. One of the course topics is generalization of the effects of interventions, and they know that one measures generalized skill by asking for performance in a variety of contexts that are superficially different but share some conceptual common elements. They also learned from a course reading that one of the most effective tools for generating generalized performance is to expose people to just such a sequence of different contexts and provide feedback and support. After a number of exposures the "aha" experience kicks in, and then people recognize all subsequent similar contexts and utilize the conceptual skill appropriately.

When I explained that my teaching strategy in the class was based on this idea from the reading, there ensued one of the most poignant discussions I've ever had in a class. The best student in the class said that seemed like cheating; if you have seen some examples of the use of a concept before the exam, then performance on the exam (even on different particulars) is like regurgitation. She urged that I stop giving students any exposure to exam-like examples ahead of time. Another fast-learning student chimed in that he agreed completely. There was a long silent pause. Then another student finally said that she really liked explicit teaching for generalization; she urged me to keep my current practices. She said she needed that kind of sequence to acquire the skill, and she was unhappy with classes in which the instructor expected that all students could form a generalized understanding from one simple exposure to a concept. There followed a lively give and take among the students, with several others joining in with acknowledgments that they were not superfast learners and they appreciated help in getting the bigger picture. Right in front of my eyes the class was playing out the conflict between identifying and stretching the top end and making sure that a large percentage of learners will retain generalized understanding of course concepts. I was glad the period ended while they were well into the conversation; I had no ready benediction to resolve the issue, and I could quietly escape to my office to sort through what I'd heard.