Sunday, November 8, 2015

Guest Blogger: Multiple-Choice Question Guidelines

  Small Professor Stockmeyer

Our Guest Blogger this week is Distinguished Professor Emeritus, Norman Otto Stockmeyer, who retired last year after teaching at Western Michigan University Cooley Law School since 1977. He also taught as a visiting professor at Mercer University Law School and California Western School of Law. Otto taught principally first-year courses (Contracts, Criminal Law, and Research & Writing) as well as Remedies. He received the top teaching award at Cooley Law three times and was voted National Outstanding Professor by Delta Theta Law Fraternity International.

 Multiple-Choice Question Guidelines

Law school professors and academic support professionals should use multiple-choice questions for assessment and testing purposes.  After all, our students will have to take and pass a bar exam with a full day of multiple-choice questions.  It stands to reason that their chances of passing will be enhanced if they have successfully taken myriad multiple-choice tests in law school.

Going one step further, I submit that our multiple-choice questions should reflect the style and format used on the Multistate Bar Exam.  The MBE professionals know more about multiple-choice methodology than we do.  And if we want our tests to mirror the MBE, we should adopt the MBE’s question-drafting practices.

The following guidelines are derived from a 2008 article in The Bar Examiner, published by the National Conference of Bar Examiners, and an examination of MBE questions released since the examination was redesigned in 2005.

       -     Use one question per fact pattern.  Do not piggyback multiple questions on         a single fact pattern.

-          Make fact patterns as concise as possible.  Do not include extraneous facts unless fact discrimination is the skill being tested by that particular question.

-          Make fact patterns realistic and free of bias.  Use genderless characters to the extent possible; otherwise equalize the number of men and women in your questions.

-          Identify characters generically, rather than by names or letters. (“A buyer agreed with a seller…” rather than “Able agreed with Baker….”).

-          Include all facts in the fact pattern.  Answers should not introduce additional facts.

-          Provide four answers for every question.  More choices add complexity with little appreciable improvement in reliability.

-           Avoid compound answers (“A and B, but not C”).  (Besides, students hate these.)

-          Do not use “all of the above” or “none of the above” answers. (Ditto)  Every question should have one, and only one, indisputably correct answer.

-          Distribute correct answers randomly.  Amateur testers tend too often to place the correct answer in the C or D position.  Savvy students pick up on this.

The overall goal of these guidelines is clarity, making sure that we are assessing substantive knowledge and legal reasoning, rather than reading comprehension.  Making questions easier to read does not make them any easier to answer.  It just makes them better questions.

In conclusion, multiple-choice tests can be a reliable way to evaluate knowledge and analytical skill.  And researchers have found that test familiarity improves student performance on standardized tests.  So using MBE-style questions can heighten the effectiveness of our tests, as well as enhance the performance of our students.

(Readers interested in Professor Stockmeyer's use of multiple-choice quizzes in a first-year course are invited to read his article on “Using Multiple Choice Quizzes” in the January 2011 issue of The Learning Curve.  It is available through SSRN at

Bar Exam Preparation, Exams - Theory, Guest Column | Permalink


Post a comment