Saturday, May 22, 2010
Most of us remember the days when law school exams came in one shape and size: 100% of the grade; closed book, one day/time in one classroom, handwritten in blue books, and all essay.
Today, however, the shapes and sizes vary greatly.
- More exams are variations on open book: code/rule book only; own outline only; one sheet of paper; everything but a live human being.
- More exams are take home or variably scheduled: take home with several days to complete; take home with a set number of hours to complete; option to take the exam for a set number of hours on one of several days; self-scheduled exams; time and location accommodations for disabled students.
- Formats run the gamut: multiple choice; essay; short answer; true-false; court or practice documents; some mixture of these.
- More professors now have a percentage of the grade for participation, presentations, papers, exercises, or other assignments.
- And the blue book has been supplanted in part or entirely by the use of computers.
Are the changes in the law school exam positive or negative? It depends.
Open book: Proponents comment that open book exams are more realistic to what practice will be like. Attorneys will have their sources or notes in front of them as they write legal memoranda, consider strategies for client cases, and address juries or judges. Many argue that it is sensible for there to be code or rule books available rather than students having to memorize lengthy sections. Some also point to the fact that a lawyer being able to find the law is far more important than a spouting rule robot.
On the negative side, however, some express concern that open book exams encourage students to gloss the law and not really know it at any depth. If only working memory is used instead of long-term memory, they will have no recollection of the basic law later when they get to bar review and practice. Others are concerned that open book exams do not really assess learning unless the professor has carefully designed application questions rather than pure information questions.
Open book exams cause some traps for students, especially unsuspecting 1L's. Students recount stories of not studying as thoroughly because they could "look it up during the exam" and then finding there was not enough time to do so. They also talk about time management problems because they felt compelled to look up everything to be certain even though they knew the answers. Other students remark on their wasting inordinate amounts of time before the exam tabbing books for what turned out to be non-essentials.
Variable schedules: Proponents argue that more flexible scheduling can allow the professor to test students in differing formats than the one place/time exam with a strict time limit. For example, the professor might ask for a memo, brief, court document, or client letter as the answer format. In addition, proponents argue that answers are better analyzed, more organized, and better written when multiple-day take-home scheduling is used.
Certainly allowing disabled students to take exams with extra time as an accommodation is an important improvement in exam procedures - as is letting them have quiet rooms, readers, or scribes. Logistics need to be carefully worked out, of course.
Letting students choose which of several designated days to take an exam at the law school with a set time limit on the day also seems sensible. By picking up the exam and returning it to a proctor under time-stamped or clocked conditions allows for fairness with flexibility. This improvement takes some of the difficulty out of exam schedules for the upper-division student who would have more exams in a series of days than a classmate. It lets a student decide when she feels ready to take the exam.
The time-limited take-home exam (for example, complete within 4 hours after the exam is opened) is manageable. The greatest risk here is that the student will be tempted to break the honor code and actually spend longer than allowed.
Personally, I worry about take-home exams that run over multiple days. First, they often do not consider the accommodations for disabled students; a take home exam that is given for 4 days means that the student with double time has to plan 8 days to work on it. Second, professors often give take-home exams that stretch far beyond the designated exam day for the course, thus encroaching on the intended study days in the schedule for the next exam (especially where 1L students are concerned). Third, students are faced with the reality that many other students will use the maximum possible hours to take the exam and they fear they must do the same to compete. Fourth, professors who tell students that they only need 4 hours to take the exam over the 4 days are usually woefully incorrect about how long the exam will take the average student. If the professor truly thinks it is a 4-hour exam then she should limit the time for taking it or give it at the regularly scheduled time.
Self-scheduled exams have an appeal for students so that they be autonomous in deciding what day and time to take each exam for each course. I have experience with this system at a small liberal arts college. However, it can be a logistical nightmare as the student body and course enrollments increase. And it depends on a strong honor code system to work.
Format changes: No doubt some flexibility away from all fact-pattern essay exams is a plus because different course material may lend itself to different question formats. When I give exams, I mix formats for different kinds of assessment.
In jurisdictions where the MPRE will be required, professional responsibility multiple-choice questions may make perfect sense. Some faculty will argue that multiple-choice should be used for MBE subjects as well. But what about the state bar essay questions? What about the performance exams given in various states? Do they require us to rethink our testing formats as well? Where is the balance between "testing to the bar exam" and assessment for law students?
I think we need to be careful to make the decisions on sound assessment reasons rather than devotion to the bar, hunches, or our convenience for grading. Here are some thoughts:
- Writing good multiple-choice questions is not easy. Training may be necessary for us to avoid poorly crafted questions. After all, most faculty do not begin their careers with test construction expertise.
- The styles of multiple-choice questions used by faculty are all over the map. They often look nothing like MBE or MPRE questions. If the justification is to prepare students for these bar exams, then the questions need to mirror the bar formats. Otherwise, the questions should be tailored to the course material and assessment issues.
- Professors who have honed their multiple-choice questions over several years tend to guard their question pools (once found to be valid and reliable) so they do not need to write new questions. However, because each professor tends to write her own style of questions, students are blindsided if the professor does not release at least some practice questions for students beforehand.
- Without someone in academic affairs monitoring the formats used by faculty, it is all too possible that a section of the 1L class may end up with no essay exams at all. And, I have talked to 2L and 3L students who have found the same because of the mix of courses in a semester. That unforeseen result suggests that we believe that there was no merit in the fact-pattern essay. Do we really want our students to have limited essay experience?
- Word limits and page limits can arguably assist students in more concise exam answers. However, we need to be careful that these limits represent what a student can write concisely as opposed to what a professor who has expertise can write concisely. And we want to make sure that these limits are appropriate to the assessment goals for our questions and not just convenient for grading.
I am lucky because my elective courses have relatively low enrollment caps. I still give comprehensive essay and short-answer exams that require my students to write a great deal. Because I do two reads of each exam (one for initial scoring and one for consistency with scoring on all papers), I create some burdens for myself. I understand the temptation that would exist to change the format if I had large classes of students. However, I hope if that day comes that I will weigh new assessment formats carefully and not lean toward my own needs for simplicity or convenience.
Multiple grades for a course: Many students tell me that they appreciate classes that do not have 100% of their grade dependent on the final exam. However, they often tell me that it frustrates them when professors give them details for those extra projects or presentations near the end of the semester (usually referring to the last 2-3 weeks). In some cases, professors cannot give out information earlier because the project cannot be completed before certain material is covered in class or themes emerge. In other cases, however, it would certainly help well-organized students to be able to plan their work over multiple weeks when they have several courses with projects.
Participation grades trouble some students because they are not "talkers." In my seminars, I designate part of the grade for participation (usually no more than 20%) because I want a seminar to have discussion and not turn into a lecture course. In addition to the usual class discussion, I provide students with opportunities to discuss websites for current items in the news so they can plan their comments ahead of time. Another option could be electronic discussion boards. Throughout the semester, I caution students to remember their participation points and not to "forfeit" them.
Computers and blue books: A few years ago, students would sometimes express concerns to me that their typing skills were not fast and accurate enough to use the computer for an exam. I do not hear that concern very often any more. Now I find that students admit that they do not have the cursive penmanship background to handwrite an exam. As professors, we tend to take that skill for granted. There have always been law students with legibility problems, but today it is far more a problem of actually not having used the longhand method since they were children. Some tell me they were never taught cursive in their entire lives and can only print! (There has been an interesting discussion on the legal writing listserv recently about this very issue.)
Typed exams certainly are faster to read. Having had several bosses with terrible handwriting over the years, I am never phased by student blue books because I can decipher almost anything. As a result, I do not think that my own students missed getting points because of handwriting. However, I can see that it could be an issue. And, if they are printing rather than using longhand on an exam, it is likely to be slower than typing.
There seem to always be a few students whose computers crash and who end up having to complete the exam by hand. The stress and anxiety are usually huge. And for most of them, they have no idea what they were typing before the mishap! Those who use scrap paper to organize answers before typing are less fazed by these problems because they can quickly get re-oriented.
The variations used today really do result in the "it depends" response. Assessment comes with a myriad of decisions to make. The quest for balance needs to be carefully thought through by each professor for each course. (Amy Jarmon)
Friday, April 30, 2010
I would like to thank Ruthann Robson, Co-Editor of the Constitutional Law Prof Blog, for alerting us to her April 22, 2010 post on that blog. Ruthann is Professor of Law and University Distinguished Professor at CUNY: Ruthann Robson Profile.
Her post lists numerous hints for professors as they draft their exams. Number 12 in the list mentions that faculty may wish to ask ASP staff for support when working on end-of-the-semester exams. The post also gives a nice compliment to David Nadvorney and his ASP colleagues at CUNY. The full post can be read here: Constitutional Law Exam Drafting. (Amy Jarmon)
Tuesday, January 13, 2009
My son's goldfish, Max, died yesterday. Ben never seemed all that interested in him after the first few days and we were happy that we got our 13 cents worth of finny fun from him in that time alone. Max was with us for about 6 weeks, but yesterday he was floating sideways in his little tank and was disposed of with all the pomp and flushing such an event requires. But we didn't tell Ben last night and we didn't really think he would notice. We were wrong. This morning he came running into our room, "where's Max?" Well, um, we stuttered, Max was sick and he couldn't swim anymore, but we can go to the pet store and get a new Max over the weekend. Well, that was the wrong answer. "I want the real Max back. I don't want another Max; if I get another fish, he will have to have a new name because he will not be the real Max." Then the tears--big, tragic, three-year-old tears--he was really and truly heartbroken. We were surprised and a little ashamed that we assumed, wrongly, it wouldn't have much of an impact on him.
The first years here got (most of) their grades on Friday. And I have seen my share of big, tragic, 22-25 year-old tears. I was a little baffled when I realized that some of these students were crying about B's. We have a B- grading curve here, so B's are actually considered above the pack-certainly not the kind of grades that will get you into academic hot water later on.
I was as tempted to be dismissive of the students' angst over these grades as I was over the death of the 13 cent fish, but I realized that I had to take a step back and really listen to these students before I decided that they were being overly histrionic. Some of these students have always been A students in prior educational settings; some have scholarships that are contingent on a certain GPA. Either way, they have always defined themselves as smart, "good student" types. To these students, a B can really shake the foundation of their self-definition--their very being.
We in Academic Support know that somewhere in the first year of law school, a student can get separated from the person they were--and the goals that they had--before they came to law school. Grades that are unexpectedly bad (even if not objectively bad) can sometimes be the nail in the coffin of a student's prior persona. This is scary stuff, especially when students are looking at two and half more years of school.
I find that reminding students to keep the grades in perspective helps. Students need to know that while the grades are important (I can't tell them that the grades are not important, would you trust me if I did?); they are only important in their context and no other. Your law school grades are not indicative of your worth as: a person, a son, a daughter, a husband, a wife, a parent or even a fish owner. You will most likely go on, not only to finish law school successfully, but to be a good lawyer. Your grades are only indicative of how well you were able to communicate your skills in a particular area of law, to one professor, in a two to three hour period, on one day.
This is not to say that I don't offer advice and guidance on how to do it better next time-mere comfort without future planning is an incomplete response to these issues. I do a post-mortem with students on the exams that includes asking: how did you study? with whom? how did you feel the morning (or evening) of the exam? how did you spend your time during the exam? did you run out of time? and most importantly, have you spoken with your professor? I offer appointments throughout the semester at regular intervals and exercises that can be done now to help get ready for spring exams. And above all, I offer tissues and chocolate.
In the end, my response needs to be the same for any student who is disappointed or even shocked by their midterm grades. Even if those grades will not effect their academic standing down the road, a student's human standing is just as important.
As for Max, may he float in peace. (ezs)
Monday, November 24, 2008
Did you miss the e-mail on the ASP Listserv sent by Barbara McFarland regarding the NCBE handout on drafting rules for multiple-choice questions from Dr. Susan Case, Director of Testing for NCBE?
Several common techniques used by law professors in composing multiple-choice questions are specifically mentioned in the drafting rules. Dr. Case is currently working on an article for the Journal of Legal Education on this same topic. I know that all of us will be interested in reading the article when it is published.
You can read the PDF file for the Dr. Case's handout here: Download multiple_choice_drafting_guidelines_by_s_2. Case of NCBE.pdf . (Amy Jarmon)
Wednesday, May 7, 2008
With apologies to Tina Turner and anyone who does not appreciate the beauty of ‘80’s music. But really, what does luck have to do with exam taking? When I send my students off to take their exams, I feel like they want me to say good luck (or break a leg for the undergrad theater majors). But I don’t want to. Why? Because after all the time and effort they have put into studying, I am not sure luck has much to do with it and I don’t want students to feel that their performance on the exams is out of their control.
I have a student who has been coming to see me everyday since classes have ended; she says that I am her study group. She has worked exceptionally hard at answering old exam questions under test conditions and we go over them everyday. We have discussed reading questions carefully, outlining before answering, issue spotting, completeness of answers and organization. We have not discussed rabbit’s feet, numerology, astrology or the idea of setting up an alter to the gods of Constitutional Law (whoever they may or may not be) in the exam room (not to mention that proctors tend to frown upon lit candles during a paper exam). Today is her Con. Law exam and as she left my office to do that last minute read through of her outline, she said, “wish me luck.” And I said no; she looked crestfallen, but then I explained my dilemma.
What is the right thing to say to my students before
sending them to the lions, I mean into exams? “Go get ‘em tiger,” seems glib and condescending. “Show ‘em what you got,” is also glib with
hints of stripper inappropriateness; and “Hrrr,” the pirate warrior cry seems
This is not to say that there cannot be any luck involved. I have been lucky every now and then on exams: like the time I had a friend contemplating a surrogate parenting arrangement in Massachusetts just prior to the bar. I did a bunch of research for her and lo and behold it was a question on the bar a few weeks later. It happens, but you cannot rely on that luck when you can’t possibly know it will happen until you are actually taking the exam. You need to be prepared for the questions you don’t happen to know the answers to already and that involves knowing the law and answering the questions so that the professor agrees that you do.
So, what I said to my student today was, “go show the professor what you know and organize it so he/she knows that you know it well.” And then I added, “I would say good luck, but you don’t need it because you have worked hard to know the material and you know how to take this exam.” While this is bit longer winded than a simple, “good luck,” I think it was a better way to prepare this student for the exam-by putting the power to ace it in her hands and not someone else’s. (ezs)
Thursday, December 27, 2007
I remember Januaries.
They begin with the AALS Conference where most of us show up to share ideas, eat too many cookies, scurry through the Thompson-West exhibit getting our cards initialed so we get the free gift and qualify for the big drawing, and ask, “Where is [fill in blank] ... did she retire already?” Then those of us who can show up at the pre-dawn (well, it always seems like that anyway) Academic Support Section overpriced breakfast meeting near the end of the week to ask each other, “Who’s hosting the summer meeting(s) this year?”
The following Monday we all return to our offices to welcome the students back for the spring semester. (Students in the northeast, as I vaguely recall, often return to snow.) All of them are asking the same question: “When do we get our grades?” The wunnelles are asking, “If my grades are horrible, can I get a refund on my spring semester books and get my tuition back?” Some have made New Year’s resolutions to study more efficiently, or visit the pub less, etc., etc.
The long wait for grades ensues. As they trickle in, so do the students – to make appointments with either the Dean of Students or the Academic Support Director (or both). Some drop by to offer gratitude, but most arrive with an array of emotions ranging from disappointment to shock – a few with anger. (I remember one student who arrived with her mother. They both explained that the student had graduated at the top of her college class, had an IQ in the genius range, and – most importantly – had several lawyers in her family (not Mom). The visit was to inform me that there’s something seriously wrong with a school that can’t figure out that she should be at the top of her class – her highest grade was a C. She withdrew.)
But this time of the year is when Academic Support professionals can do some of their most effective work – many students are now willing to admit that what you told them at Orientation really did apply to them.
If you’re relatively new to Academic Support and fortunate enough to be able to attend the AALS Conference, that’s a question to be asking your colleagues –“How can I be most effective in January for the students who have disappointing grades?” Search out the “veterans” and find out their (open) secrets. As weird as January is around a law school, it can be a very productive time for the Academic Support staff!
Me? No AALS this year. I wish I could! But the distance between New York and Montevideo is about 5,500 miles, the air fare is prohibitive, and I just compared the weather report for January in New York to January on Pocitos Beach in Montevideo. (Remember, it’s summer in South America in December.)
Also, the academic support I’m providing to students of Concord Law School via cyberspace is of a different variety – for me it’s limited to extensive (written) exam-answering improvement advice, including (unlike yesteryear in law schools with buildings) explanations of the underlying law when appropriate. I spend fifteen to twenty hours each week at this pursuit, reviewing essay answers that range from beginning students’ awkward attempts, to crystal clear, concise, excellent, lawyerlike answers. My comments are composed of footnotes to most every issue discussed by the student, followed by “overall” suggestions on how to improve. All of my work is reviewed by the professor teaching the class (and modified if necessary) before being sent to the students.
Of course this is time consuming. After reviewing many hundreds of exam questions (Torts, Contracts, Criminal Law, Property, Evidence), I still spend at least thirty minutes (usually longer) on each one. That’s what makes this type of feedback both (a) very valuable for the students, but (b) virtually impossible for a one-person academic support office at the typical law school-within-walls to handle. But I’ve got to say – this is something I’ve always believed students need: practice, practice, practice … with substantial feedback consisting primarily of encouraging positive improvement advice.
So even though I don’t get to see the smiling faces of the successful students, I suppose that’s balanced somewhat by the time not spent with … well, you know.
I have to admit that “going to work” (in my living room) in attire ranging from pajamas to blue jeans is a plus, too.
Enjoy AALS – I will truly miss a week with you. (djt)
Wednesday, November 28, 2007
First things second.
Spotlight time. Presenting ... ALEX RUSKELL. Alex took over leadership of the Academic Success effort at Roger Williams University School of law this academic year. From all reports, he's doing a super job!
Before this year, Alex served as the Director of the Academic Support Program at Southern New England School of Law, and before that, Associate Director of the Legal Writing Center at the University of Iowa College of Law. In his earlier life, he litigated in Boston, focusing on securities and corporate non-competition agreements. He has also served as General Counsel for a mid-size publishing company, Associate for a large oil and gas firm, and as an Assistant in the Texas Attorney General’s Office of Environmental Crimes.
His academic background is varied — and thus well-suited to academic support! He holds an M.F.A. in Fiction from the University of Iowa Writers’ Workshop, an A.L.M. in English from Harvard University, a J.D. from the University of Texas at Austin, and a B.A. in English from Washington and Lee University.
Before practicing law, he taught in a Russian orphanage and counted otters for the Idaho Department of Fish and Game. Both of these resulted in several articles, printed in The Tampa Tribune and many other publications.
Alex frequently presents at writing conferences and symposiums across the country, most recently at the 2006 AWP Conference in Austin, Texas, where he sat on a panel questioning the continuing vitality of the American novel.
Now, how does this tie in with "sharing"? Alex gave me permission to post his latest exam-answering advice to the RWU SOL students. It's terrific. Here goes . . .
Monday, September 24, 2007
All of us who are blessed with sight have some degree of visualization capability. Often, accessing that right-brain function can bring clarity to a text-based problem that borders on incomprehensible during the stress of an examination.
Do you tell your students to draw diagrams during the pre-writing stage of exam-answering? Consider showing them how. Of course, the illustration above (based on a standard Blueacre & Whiteacre real property problem) has more actual words in it than the diagram the student would make in the rush of an exam session – these words are for illustrative purposes here. If you demonstrate how to convert a Property exam problem into a sensible (literally) representation, be sure to stress that abbreviation is essential! (djt)
Tuesday, March 20, 2007
In all the time I've written for this blog, I don't think I've mentioned that my husband is an environmental lawyer. His main area of work is clean air and climate change. He works for an environmental non-profit (and for those of you doing the math, that does make us the lowest paid attorney couple in all of Massachusetts) where he spends a lot of time discussing global warming (and no, my nickname is not Tipper).
Anyway, as a result of being married to "Captain Environment" (which is what I call him when my pleas for a minivan are rejected), we are big recyclers. We are the people who drain the juice boxes and put them in our recycling bin. I've tried looking for the recycling symbol on the little straws too, but so far no luck. Anyway, to do my part in reducing global warming, I am recycling an older blog entry (or as the Car Talk guys put it, "an encore presentation") on exam writing. Why? Not because my brain is empty (although my "fill brain" light did go on recently), but rather because I find that I am printing this entry out for students at least twice a day.
So here it is, just in time for spring exams:
Exam Writing is Different:
Exam writing is different from first year legal writing because, while the usual 1L legal writing class teaches writing as a craft, exam writing is a skill. What does this mean? Legal writing for class is something that students have more time to work on; in addition, almost the entire exercise is about students' organizing their thoughts and choosing their words carefully. Exam writing, on the other hand, is a formulaic way for students to show their professors that they not only understand the material but can also use it.
If legal writing is a painting, exam writing is a photograph. Exam writing must be done quickly and accurately; and while it must also be done with some creativity and is worth the same 1000 words as a painting, it has to be more stark and realistic. Some of the best 1L legal writers will not do well on their exams because they cannot leave an answer in its raw form. Rather, they need to hone their thoughts and organization until the answer is perfect; and frankly, they do not have the time to do it, so their grades do not reflect their knowledge of the subject or their writing ability.
So how do we prepare students to let the raw answer be and move on to the next question? I do the math with students, trying to make the point that one fabulous answer on a four part exam is still unsatisfactory, while four reasonably complete and cogent (but imperfect) answers will suffice. I teach them the exam mantra: "the issue is...the rule is....here we have....therefore..., next." I think I hear students muttering this during exam week, or I'd like to think that's what they are saying as they pass me in the hall....
I also advise taking old exams under exam conditions (timed without notes and books) so students can figure out their ideal ratio of time to organize vs. time to write (usually about 20%/80%). I talk about organizing answers by party (good for torts, not civil procedure) or by transaction (excellent for contracts, but not for torts), etc. I give the old tennis analogy (you still need to follow through after you’ve hit the ball) as a way of reminding students to include counterarguments and defenses. I, personally, had to write the word DEFENSES at the top of every exam in law school lest I forget to include them.
After working in academic support for a while, I have concluded that a student’s performance in the first year legal writing class may not be an indicator of how well that student will perform on exams. Oddly, only about half of the students I see find this comforting. (ezs)
Thursday, November 30, 2006
At this time of year I am inundated with students who are stressed about upcoming exams and in particular the multiple choice format that many of these exams will take. I give them all my standard advice: practice, practice, practice. Why? Well, certainly not because when they don’t get this as the punch line to the joke, “how do you get to Carnegie Hall?” it makes me feel really ancient, but rather because it (hopefully) works.
Practicing multiple choice questions from a variety of sources gives students more fluency in the language of the topic they are being tested on. I analogize (and I analogize a lot, it is a lot like, whoops…) it to moving to a new city and doing the crossword puzzle in the local newspaper. When I lived in New York and did the puzzle (on Mondays and Tuesdays only), and the clue was, “off-white,” the answer was usually “ecru.” Here in Boston, (where I can sometimes last until Thursday) the answer is, “beige.” How did I realize this? The same way you get the Wang Center.
Also, sometimes (and I know this would a very rare circumstance) doctrinal professors actually get their multiple choice questions from other sources. This means that the language used in the questions may differ somewhat from the language used in class. This really can throw students off (especially ESL students), because they are not accustomed to seeing the issue raised in this slightly different terminology. Practicing questions that come from many different sources: study aids, on-line lessons or bar review books, can help a student see the “disguised” issues more frequently.
Another strategy I offer students for dealing with multiple choice questions is to read the question with the answers covered. Then, they should come up with an answer in their heads, match it to one of the options and move on (yes, you do need to uncover the answers at some point). I think this is sound advice based on my experience as the sole food shopper in my household. (I think maybe my experience in Academic Support has played a small role, but I am not sure.) Here’s why: when I go shopping without a list, I find that I end up with many items I don’t need, fail to get some items I do need and invariably spend more time and money than I intended.
Multiple choice answers use the same marketing approach as the supermarket: make it look good, and they will buy. Almost every answer on your exam will look plausible if not downright compelling (like that $1.00 giant tub of Fluff sitting in my cabinet). That is the nature of the beast. If you go shopping for an answer, chances are you will be pulled in by the display. If you have a list (or the answer in your head) prior to your shopping trip, you are more likely to make the right choices.
And finally, like grocery shopping, you should never go to your exams hungry. (ezs)
Wednesday, September 27, 2006
Finally, my students who found themselves outside of good academic standing (would that be academic sitting? lounging? lying about?) have come to see me with their exams from last year (also known as, “the reason why I have to see you…..”). So now what?
I just had a student come by with three exams from last year. This student had not talked to the professors who graded these exams except, evidently, to request a copy. Copies in hand, the student came to me looking for the reasons why his/her grades were below par. One professor kindly included an outline of the major issues and how they were to be resolved as well as a copy of the exam itself, the others did not. None of the professors included their multiple choice questions or the student’s score on them.
So, I read the exams. I tried to decipher the various marks left by the professors. Some were easier than others, like the “?” or the “No!” But most of the marks were mere numbers to be added or subtracted from the student’s total for scoring purposes. So what else is there to do but call in my personal (that is: living in my head) Crime Scene Investigators.
First, I called in the spot the issues (“STI”) expert. The spot the issues expert examined whether, thanks to the outline provided by the professor, the student found all the possible issues to be discussed. Then, I called in the correct rules applied (“CRA”) guy who looked for whether the student had, in fact, not only spotted the issue, but actually used the correct tool to get a plausible answer. Finally, I consulted with my use the facts (“UTF”) expert who looked at whether the student used almost every fact contained in the question. These experts were only able to operate on this one exam because the professor had left me with some clues to work with. In the end, I had to ask the student to talk to at least one of the other professors.
As to the exam I could investigate most fully, I diagnosed a “kitchen sink” type of crime, that is: the student used every rule they knew and contained therein was the correct rule, but because there was so much superfluous analysis required for the issues that hadn’t really arisen, the real issues received short shrift.
I will have to file the other exams away until the student sees the other professors. Hopefully, in ten years, someone from “Cold Case: ASP” will be able to figure out what went wrong. (ezs)
Friday, May 12, 2006
About a week ago, I had a student come into my office after her Property exam. She tearfully told me that she didn't finish the exam because she had only left herself twenty minutes for the last question (where the exam had apportioned one hour). Nothing I said, or could have said (I now realize) made her feel any better. She left my office weeping. About thirty minutes later I left the building on my way home and saw an ambulance out front. I was frantic--had my student succumbed to the Property exam? I actually looked inside the ambulance to find out. It wasn't her and I was greatly relieved, but I still felt that I offered her no comfort.
And that bothered me. I have both described and criticized as being very maternal in my support style. Under rare and appropriate circumstances (and with consent and all sorts signed waiver forms), I will hug. I always have candy and tissues. I have helped students find bridesmaids dresses and doctor referrals, but I could not help this student.
I told my tearful student that it was not the end of the world; but to her it was. I told her that knowing her, the rest of the exam was great and what she did manage to write for the last question would probably have been another student's best effort given all the time in the world. She wasn't buying it. She asked about having to repeat the class if she failed. I truly do not believe that she failed, but I answered her questions after prefacing them with my belief that the information provided would not be useful for her. She stormed out of my office more upset than when she entered.
Basically, short of tap dancing (which would have been an ill-advised attempt at humor); I had used up my repertoire of "it's ok" tricks. My firm belief in her intelligence and preparedness wasn't enough and that is okay. I was more worried that since this was the first exam for first year students that her belief in herself was shot as well. I haven't seen her since and I don't expect to. I doubt she realizes that I am still worried about her or that I scared some lovely Boston paramedics by sticking my head in their ambulance because I thought she might be that distraught.
Today, another student came to me and complained about the same exam. I asked him if he finished it, and he said yes. His problem: the exam was too hard. Why? Well, he said that he had been surprised that there was a whole essay question on future interests on the test. He had believed, based on his empirical analysis of prior exams that future interests only appeared on the multiple choice part of the exam. I assured him that if he had studied well for the multiple choice that he was also prepared for an essay. He disagreed. So, I told him, in contrast to the first student, that the singer they voted off of American Idol on Wednesday was far more surprising than a future interests question on a property exam (in fact it was downright shocking, I thought the guy was a shoe-in, but that is another story for another blog).
In the end he wanted to know if I thought he should ask the professor if was going to fail. I told him that the professor had probably not graded the exams yet and that his answer was likely to be, "I can't tell you yet;" and that would not provide any comfort at all and might, in fact, be more upsetting. Again, I doubt I'll see him again.
In the end, I think I my reactions to these two students indicate that I have achieved complete ASP motherhood. And in that vain, I wish myself and all the rest of the ASP mothers out there, a Happy Mother's Day. (ezs)
Monday, May 1, 2006
Here in the still chilly northeast, exams start later this week. Since part of any ASP office's duties include some exam time cheer-leading, I had seriously thought of wall-papering our bulletin board and office area with notices that read, "Good Luck on Exams from the folks at ASP!!!!!" (I do tend to use way too many exclamation points!!!!!! So I'm enthusiastic; is that a crime?????????) But then I thought better of it.
Certainly I have not taught my students that their exam performance has anything to do with luck. Sure, there can be some luck involved; as in: "wow, I am sure lucky that last question had to do with adverse possession because I knew that one cold." There is absolutely no luck involved in knowing it cold.
Knowing the material comes from doing the reading, going to class and outlining. Knowing the material comes from studying the outline and spending the time to study effectively and efficiently (Dan is right, sleep is very important!). So I have revised my sign. Instead of wishing my students luck, I wish them this instead:
Good Organization of your answer!
Good Timing on the exam so you can finish it all!
So essentially, I wish them all Good I OAT. I think it may catch on; and it may lower your cholesterol as well. (ezs)
Wednesday, April 26, 2006
Professor Rogelio Lasso of the John Marshall Law School has put together some great advice for taking essay exams in "How to Write a Good Law School Exam." Particularly helpful is his advice on how to develop and outline an answer for a typical essay question. Best of all, he includes an example question with a model outline of the first part of the answer to illustrate the techniques he suggests. (dbw)
As the semester comes to an end, I have been telling most of my students (the ones who have been seeing me based on their midterm grades) that they have earned their wings and it is now time to fly. These students have done all the work and they are ready for exams.
As part of this conversation, I always ask these students if they feel they are in a better place than they were last semester at this time. And the answer really should be “yes” regardless of whether I have worked any ASP magic on them or not. Why is this? Because last semester at this time, these students had no idea what law school exams would be like. They did not know (which is not to say it wasn’t knowable) the level of depth each answer required, nor did they understand that spending the time to outline your answer before writing it, was analogous to priming the exam pump. But now they do--or should.
I think of this “exit pep talk” like the first comforting words of Dr. Spock’s book on babies, “You know more than you think.” Or actually, sadly, more like those old cigarette ads, “you’ve come a long way, baby.” (This, of course, is language one would never use with students but I think it really captures the moment.)
Second semester first year law students (we’ll call them “SSFYs”) are far more savvy about exams than first semester students. I imagine that even those students who performed well on the first semester exams have a better sense now of why their performance was superior than they did in January. What do SSFYs know now that didn’t know then?
SSFYs know how to write a case brief that is shorter than
SSFYs know how to create a rules-based outline.
SSFYs know that Prof. Tonsing’s post on “practice” is dead on and have practiced hundreds of multiple choice questions and a fair number of essays per class, or plan to, in preparation for exams.
SSFYs know that the analysis is the most important part of the essay exam answer.
SSFYs know that the potential answers to a multiple choice question are all intended to look compelling and to not look at them before having an idea of the answer in their heads.
SSFYs know that their grade is not personal.
SSFYs feel more at home here.
SSFYs know (most importantly) they will live through exams and that the sun will indeed rise the day after they are over.
Did I teach them all this? I’d like to credit for it, but I cannot. All I may have done is make students aware that they know more than they think. (ezs)
Thursday, April 20, 2006
In the fall I wrote about having an “exam plan,” that is, a time management strategy for getting through the reading period and exams. But I have been thinking lately about having another sort of exam plan in addition to time management. I have been asking students to come up with an exam strategy, that is, a method they plan to employ while taking the exam to maximize time efficiency and organization of their answers.
I know it sounds a little like a carnival tonic, but here’s the idea: have a plan of attack for each subject. For example, a student could plan to tackle a Torts exam by creating a chart of the parties v. the other parties (after reading the question, of course), listing the potential causes of action that arise between each set of parties and then use this chart as a check list of issues to deal with in their answer. It would probably look something like this:
A v. B-negligence, battery
A v. C-negligence, tortuous interference with a K
A v. D-nuisance
B v. C-assault, etc.
In Contracts and Civil Procedure the strategy could be more chronological. Think of it as an obstacle course, what are the hurdles to be cleared? In Contracts, a student needs to deal with formation, and then the terms of the contract, and then if there has been a breach, and finally if there are damages from the breach and then remedies. Don’t forget to think outside the box and cover non- and quasi-contractual concepts as well. In Civil Procedure, a student might have to think through jurisdiction, complaint, answer, 12(b) motions, joinder, discovery, summary judgment, etc.
I have advised students that having a planned course of action before even starting the exam is a good idea. If you have a closed book exam you might want to memorize a very skeletal checklist of potential issues and immediately write it down on your exam paper. In an open book exam, bring your “obstacle course” with you. I also have advised students that this kind of planning can be done collaboratively with a study group (as opposed to outlining which should be done alone).
Personally, I had to write the words, “counterarguments” and “defenses” at the top of every one of my exams. I guess I had a problem with being wrong. (ezs)
Wednesday, April 19, 2006
For an exceptionally thorough explanation of strategies for answering multiple choice questions, I must again recommend that you check out Vernellia Randall's advice, this time to bar takers. She provides detailed explanations of question types and analytical methods for addressing them. Her advice is as relevant for law students as for bar takers. Take a look. (dbw)
Monday, April 3, 2006
And I don’t mean in March as opposed to May, I mean in third grade as opposed to law school.
Last week my nine-year-old took her reading MCAS (Massachusetts Cruddy and Stupid, I mean: Massachusetts Comprehensive Assessment Scale) exams, and she was truly worried about it. She was afraid these exams would define her forever as either smart or not. I tried to re-assure her that this was not accurate; after all it is my job to tell (relative) grown-ups this everyday.
She didn’t buy it.
I explained that this is more a test of how well the school and her teacher are doing in the business of third grade teaching. She didn’t buy that either. I have to wonder, if I can’t convince a nine-year-old, are any of my law students reassured by anything I say? I’ve always thought that people found me believable, or at the very least harmless (for example: people ask me for directions at least twice a day no matter what city, town or country I am in!).
I find I say the same things in different ways for different students, but basically my message remains the same: exams are an indication of how well you answered a certain set of questions on one particular day out of your whole life. And while how you do on this limited assessment is important, it should not be all defining.
Which is not to say that you should not be prepared for exams: they are not, after all, random. Students need to attend class, outline in some way, and study effectively and efficiently for these exams. Students also need to know what is expected of them on exams: not the actual subject matter per question, but the level of depth required in answering each question. Our doctrinal professors do give out this information but a number of students do not engage in this highly important dialogue until after the grading is done. A number of professors offer to give students prior exam questions and then give feedback on how the student answered the question.
This is law school gold and yet students rarely partake of the riches offered. Why? I’m still not sure. A student’s imagined exam must be far worse than the real exam but, for some reason, not being surprised by the format or the way the questions are asked is not a priority for many students. I am baffled by this because the way our school prepared the third graders for the MCAS was to have them do sample questions from prior tests. This seems to be tested and proven technology for exam readiness. So, as we ease into the lovely spring days of April, I will be chanting, “old exams, old exams, old exams” in the halls here.
In the end I reassured my daughter that she was prepared because she had practiced similar questions for weeks and done well on those. This, I think she bought, because during the three days of testing her only comment was that she was excited that they recess twice a day on MCAS days and that was “cool.” And if you are ever lost in Boston (or anywhere else), come and find me and while I may not be able to help you find your way, you will be in no danger whatsoever. (ezs)
Tuesday, January 24, 2006
We all tell our students to take a look at their exam answers after the grades are posted.
Many ask, "What exactly am I to look for when I read my old answers?"
"I thought the attached excerpt from an e-mail that I sent to all of my law students might be of help to colleagues," Professor Jarmon wrote to me recently.
Take a look at Dean Jarmon's work. I intend to post it for my students, and direct the professors at my school to this web copy of the excerpt. (djt)
Tuesday, November 29, 2005
James Kilpatrick, in his book The Writer’s Art, tells us that a writer “functions as a kind of forest guide . . . escort[ing] the tenderfoot along an unfamiliar trail.” Kilpatrick observes that this “metaphorical forest ranger would find himself in trouble if he embarked upon a hike with no plan of getting from Point A to Point B.” Therefore, he advises, “Before we begin a letter, an editorial, a brief, or a note of instructions to the babysitter, we ought to have a clear idea of the points we intend to make and the order in which we intend to make them.”
Novice writers, however, have a very difficult time accepting Kilpatrick’s advice because they cannot shake the feeling that outlining is a waste of valuable writing time. Nowhere is that reluctance more evident than in the way most students approach writing an answer to an essay question on an exam.
When a student has sixty minutes to answer a particular essay question, everything in him screams that there is no time for outlining the answer. He must begin writing – NOW – if he hopes to get everything said. He begins writing, therefore, almost immediately after reading the question. He may jot a note or two in the margin of the question and may underline a word here or there, but he is not going to waste his time preparing an outline that no one will read and that will garner him no points.
That approach, of course, misses a key point about writing: writing is a form of thinking; the written product itself is only as clear as the thinking that preceded it.
A student who understands that principle knows that she cannot begin writing in mid-air if she hopes to provide the reader with a carefully reasoned and well-crafted response to a complicated question. She understands that, even under the time pressure inherent in exams – perhaps especially under the time pressure inherent in exams – thinking before writing is critical.
Therefore, she takes time to brainstorm the points she will need to cover, rereads and rethinks the question, revises her brainstorming, and converts it into an outline of points and sub-points that follow a logical plan leading to a carefully considered and thoroughly explained conclusion.
In other words, she thinks before she writes the answer itself because she knows how writing works and how reading works. She knows that to begin writing in mid-air is to think inefficiently and to do it on the reader’s dime. She knows that unless she plans before she writes, the reader will treated to her confusing, inefficient, meandering attempt to think through the question. She knows her reader will find himself wandering in the woods, off the trail and unable to find his bearings.
She also understands that she can write as least as much by thinking and planning first as she can by simply musing in a stream-of-consciousness form. The irony for the student who thinks he hasn’t time to plan is that he will waste at least the same amount of time writing in fits and starts, rereading his answer to find where he has been and where he is headed, and staring into space trying to think of what should come next.
The primary difference between the two writers is that one spends large amounts of time inefficiently groping for words and the other spends that time efficiently designing a fully considered response. The planner takes no more time than does her colleague, but her end product is coherent and easily read. The non-planning writer, in contrast, produces no more prose than his colleague, but his answer is a photograph of his confusion as he attempts to figure out what he needs to say; so his professor watches him stumble through the thought process with all its missteps and false trails.
Worse, because the unplanned writing is likely several pages of prose, it defies any attempt by the writer to see what is missing or to cut out what is extraneous. The trees obscure the forest, so the writer misses issues and sub-issues, arguments and counter-arguments, because he cannot see the big picture and the parts’ relationship to one another.
The planner, on the other hand, steps back from her outline and makes a final judgment as to what is there, what is missing, and what should be jettisoned. She can see the gaps and the illogical arrangements, and she has not committed several pages of writing to missteps.
If we can help students realize that writing is in fact a thinking process, we can help them see that the best writing product is an end result of thinking carefully. Taking time to think using some sort of sketched outline, even during time-pressured exams, is critical to producing thoughtful prose. Thoughtful prose, in turn, is the sort of prose most likely to lead the reader comfortably and efficiently along the unfamiliar trail of the writer’s analysis; and it is the type of prose most likely to provide all the steps necessary to complete the journey. (dbw)