May 29, 2007
Grades, Participation, and Analogic Reasoning
Posted by Jeff Lipshaw
Grading at Tulane is completely blind. I taught four classes with an aggregate total of over 180 students and among non-LLM students I could never tell from the exam paper itself who was writing the exam. So it is always interesting to submit the final grades and then get back the translation sheet to find out who the numbers represent.
1. I just finished up and submitted grades for a large class. Of the reasonable number of full A grades, all but one were active contributors in the class discussion. On the other hand, I could only associate names with faces for 25% of the combined C and C- grades.
2. Of the A grade recipients whose names I could immediately associate with a face (i.e., all but one), most were not at all surprising given the cogency of class comments.
3. I did not take attendance, so I will never know if there was a correlation between attendance and grade, but I am thinking that the reason I could not remember some of the lower grade faces is that they didn't come very much.
4. My wife (an alumna of the Michigan Business School) passed along this link to a study purporting to rebut the "Jack Welch - GE" forced ranking system of evaluating employees. Here's a quote:
"The use of rankings to scale employee performance relative to that of their peers, instead of using predetermined goals, may negatively affect employees' willingness to maximize joint gains that will benefit the organization," said Stephen Garcia, adjunct assistant professor of management and organizations at the Ross School and assistant professor at the Ford School of Public Policy.
"Individuals will care less about performing better on a given task and will, instead, shift their focus to performing relatively better on a scale comparison---in other words, being surpassed in rank."
See Andy Perlman's post over at Legal Ethics Forum on whether interpersonal skills should be a factor in law school admissions. I sense there is a connection, but this is just a blog, so I will merely juxtapose rather than analyze.
5. In a new essay, "Law and Philosophy at Odds," Larry Alexander and Emily Sherwin pretty well put aside the possibility of any meaningful legal reasoning by analogy from case to case. As readers know, I have been reading Douglas Hofstadter's new book I am a Strange Loop, which ponders the essence of the self and consciousness, and finds the power to analogize at the very heart of thinking itself (the entry "analogies, serious examples of" runs over a full page in the index). My wife and I discussed this just now walking the dogs, although we digressed onto the subject of the old "analogy" questions on the SATs. Did those analogies actually test reasoning power, as though there is in fact a better analogy, or simply, as my wife contended, vocabulary, i.e., if you actually knew what all the words meant the analogy part was easy? So I posed this one: "Hockey is to goalie as baseball is to (A) pitcher, (B) catcher, (C) backstop, (D) batter." I could actually make arguments that each one is correct. Is one objectively better than the other, in the way that deductive reasoning from premises to conclusions either is or is not? Or is it more cultural, and "Family Feud"-like (recall Richard Dawson: "survey says.....!") [As usual in these discussions on the dog walks, it is not Venus and Mars, but terra firma (her) and outer space (me), so this issue was not resolved. She said the SATs would never have one like the hockey/baseball example; I said I remembered them all being like that.]
6. All of which is to say that it's probably the case that we are testing conformity, not creativity, on law school exams, but it's likely that conformity wins over creativity in the real world. Somebody just commented below on an earlier post:
That is why law school grading is broken and badly needs to be fixed. I love legal theory, unfortunately one time pressured exam at the end of the semester allows me almost zero opportunity to show that to my Profs. It also forces students to care exclusively about the end game rather than appreciating all that legal scholarship has to offer. The fact that you are met with blank stares and questions like, "will this be on the exam?" is sad in a doctoral program. This is all very fixable and would only require giving students the proper incentives.
I think that is a heartfelt and sincere sentiment, but I'm not sure I agree with its import. I wasn't looking for creativity on the exam, but nobody wrote an exam that forced me to say "wow, that is just a completely different and unique way of looking at the world of secured transactions or sales, and even though it doesn't really answer any of the doctrinal questions, I should throw out the answer key and give it an A." And I think that's the way most people expect it; they're here to learn the doctrine, and how lawyers think and operate, and the ones who like theory self-select into the appropriate upper-level seminars.
7. More on thinking in analogies to come. I hope.
TrackBack URL for this entry:
Listed below are links to weblogs that reference Grades, Participation, and Analogic Reasoning:
I'm inclined to think Alexander and Sherwin are deeply mistaken, but I'm not prepared to make the argument here (taking on Professor Alexander should not be taken lightly). And some of us may need to do our homework before your next post on analogies, so I will be so impertinent as to suggest the following:
Bailer-Jones, Daniela M. “Models, Metaphors and Analogies,” in Machamer, Peter and Michael Silberstein, eds. The Blackwell Guide to the Philosophy of Science. Malden, MA: Blackwell, 2002.
Brewer, Scott. “Exemplary Reasoning: Semantics, Pragmatics, and the Rational Force of Legal Argument by Analogy,” Harvard Law Review, 10 (March 1996), pp. 923-1027.
Fauconnier, Gilles and Mark Turner. The Way We Think: Conceptual Blending and the Mind’s Hidden Complexities. New York: Basic Books, 2002.
Gentner, Dedre, Keith J. Holyoak and Boicho N. Kokinov, eds. The Analogical Mind: Perspectives from Cognitive Science. Cambridge, MA: MIT Press, 2001.
Hallyn, Fernand, ed. Metaphor and Analogy in the Sciences. Dordrecht: Kluwer Academic Publ., 2000.
Hesse, Mary. Models and Analogies in Science. Notre Dame, IN: University of Notre Dame Press, 1966.
Hofstadter, Douglas R. (and the Fluid Analogies Research Group). Fluid Concepts and Creative Analogies. New York: Basic Books, 1995.
Holyoak, Keith J. and Paul Thagard. Mental Leaps: Analogy in Creative Thought. Cambridge, MA: MIT Press, 1995.
MacCormick, Neil. 'Coherence, Principles, and Analogies,' in his Rhetoric, and the Rule of Law. New York: Oxford University Press, 2005, pp. 189-213.
Sunstein, Cass R. “Analogical Reasoning,” in Legal Reasoning and Political Conflict. New York: Oxford University Press, 1996.
Weinreb, Lloyd L. Legal Reason: The Use of Analogy in Legal Argument. New York: Cambridge University Press, 2005.
Posted by: Patrick S. O'Donnell | May 29, 2007 7:07:56 PM
I wasn't in any of your classes, so I don't know what was on your exam or what you covered in your class. The impression I got from the earlier post was that you tried to engage your class in some of the more theoretical, though not completely irrelevant, aspects of contract law and you were met with an apathetic response. I was merely suggesting that Profs ought to give either multiple exams or an exam/paper split which would cover everything that was discussed in the class instead of a single exam, which in my limited experience, tends to be somewhat underrepresentative. I also think this would lead to a curve that is less arbitrary in the nebulous middle, since you would be increasing your sample of gradeable material and therefore eliminating some of the randomness that is inevitable when you decide which issues to put in the hypo at the expense of others, as well as some of the performance randomness (good student, bad day and vice-versa). The bottom line is I do not think the one time-pressured exam at the end of the semester is the most meritocratic option. I think it overvalues quick discursive thinking and undervalues a more contemplative approach. I think both styles of thinking are useful and both styles of thinking ought to be tested in similar proportions. I know this started as a discussion about testing theory as well as doctrine, but I think the same analysis applies to purely doctrinal exams - there is superficial doctrinalism and there is nuanced doctrinalism. While I have the floor I also also think old exams and answers should not be posted, I am curious to know how you feel about that proposal?
Posted by: Anthony Hirschberger | May 29, 2007 9:25:55 PM
Anthony: I've never had the chance (yet) to teach an upper level theory course, but I can't imagine grading it as I have my basic contracts, sales, secured transactions, business association classes. Your proposed grading method sounds reasonable for that kind of class. (I took Jurisprudence from Tom Grey, and our exam was a take-home with, as I recall, little or no time limit. I'm still pleased with what I wrote, although I took it pass/fail and never went back to see what grade I got!)
My impression is that I am not an outlier as a grader - that is, the people to whom I give high grades generally get high grades, and vice versa. After all these years, getting good law school grades on exam still seems to boil down to spotting the issues, getting the law right and doing a cogent application of the law to the facts.
I have no issue with giving out old exams, and even the grading key. In my smaller class, I gave out my class notes after each unit, on the explicit agreement that students continued to attend. Even with all of that, my exams still always create a curve, even if all the curve is measuring is the ability to cram, issue-spot, and discurse quickly.
Posted by: Jeff Lipshaw | May 30, 2007 2:44:45 AM
In many cases in private practice, though (especially in transactional practice) quick thinking on one's feet and being able to recall large areas of law to know whether a client's goal is obtainable is really the necessary skill. In that sense, law school exams do test what we'll be performing.
Also, do you think the law firm bonus structure is similar to the Welch model of evaluating employees? Finite bonus pool, and large numbers of associates trying to get a larger share of it seems to me to be very similar to that type of evaluation.
Posted by: Jenny | May 30, 2007 8:20:29 AM