Friday, September 20, 2013
As I've mentioned before, my final exams in contracts are part multiple choice and part essay. I feel a bit guilty about multiple choice questions, but since the bar exam includes multiple choice questions on contracts, students really do need practice on those. There is thus no reason for me to feel guilty unless I feel guilty because I am preparing students to take the bar exam rather than preparing them to be lawyers or teaching them the law of contracts.
Because we are doing more interim assessments these days and because I have 140 students in two sections of contracts, I have also been giving multiple choice quizzes throughout the course. I am getting fairly practiced at writing multiple choice questions that work.
I have two criteria by which I judge whether or not a multiple choice question works. First, with respect to at least 90% of the questions, I would like the correct answer to also be the most popular answer. Since I order my questions from easiest to hardest, I am not too troubled if the last few are so hard that only the strongest students can get them. But on the whole, if too many of the questions are that hard, I worry that the distinction between a student who knows the material fairly well and a student who doesn't know the material at all might be too hard to discern. Second, with respect to every question, the strongest students (top quartile) must do better than the weakest students (bottom quartile). If that is not the case, I either worry that I have created a trap that somehow captures the wary but not the unwary or that again random guessing plays too great a role in explaining the result. In any case, the question must be tweaked or retired.
But a failed multiple choice question is really a question that needed to be used as an essay instead. Multiple choice questions fail because there is not one, right answer. There may be an ambiguity in the fact pattern that could lead to differing reasonable conclusions or there might be a hidden issue in the fact pattern that students could follow down an alternative path. I just finished going over with my students a practice multiple-choice exam that included failed questions culled from past exams. I think such questions are a good way to work on both the skills needed to do well on multiple-choice exams and the skills needed to do well on essay exams.
I would call on one student for each question, and that student had to give me a defense of her answer. We then voted by clicker and discussed whey the reasons one could give in defense of the popular answers. Some answers that got some clicker votes were clearly wrong -- e.g. applying UCC rules to a service contract, but sometimes students could come up with good reasons for their choices (even if those choices were not the one I was hoping for) that would garner a lot of points on an exam designed to measure abilities to spot issues and engage in legal reasoning.