Saturday, February 16, 2019
Takeaways from “Best Practices in High-Stakes Testing” Conference
On February 7–8, 2019, the National Conference of Bar Examiners and the Law School Admission Council co-hosted a conference entitled “Best Practices in High Stakes Testing: What Legal Educators Need to Know.” The conference consisted of a welcome reception on Thursday evening, three plenary presentations on Friday morning, and a choice of breakout sessions on Friday afternoon.
After Friday’s opening remarks—which included a nod to academic support pioneer Paula Lustbader—psychometrician Gage Kingsbury explained the difference between admissions testing and licensure testing. Admissions testing is characterized by its lack of a minimum “passing” score and its need for a high level of reliability across a wide range of performances. (Here, the LSAT.) Meanwhile, a licensure exam typically follows a program of instruction, and includes a pre-identified passing score. In addition, a licensure test must possess a high reliability that the pass/no pass line is properly drawn. (Here, the bar exam.)
Both types of exams should possess not only a strong case for the validity of their use, but also the fairness of their use. Validity, in this context, means that the test is being used for an appropriate purpose. (Validity is not the same thing as reliability; reliability is the likelihood that the examination will produce consistent scores over time for all takers in the normal range of performance). Fairness, however, is more complicated. Most psychometricians agree that a test need not ensure that all subgroups (e.g. Caucasians and Latinos) score the same, but that the test should be equally reliable and equally valid for all subgroups.
Dr. Kingsbury highlighted how the term “high stakes” is frequently misused by the testing community. Almost every test is high stakes for someone; the key is to recognize for whom. Is the test high stakes for the test taker, the testing agency, the educators, the community, or some combination of stakeholders? The more stakeholders who are impacted by the results of the examination, then the higher the stakes. Moreover, a real issue arises when the examination is high stakes for one population, but not for another. For example, a statistical “terminal disaster” occurred with the “No Child Left Behind” testing model because the examination was very high stakes for the educators, but a no-stakes examination for the test taker.
He concluded with a quick tip: When crafting high stakes examinations, drafters should pay attention to how much information is being tested per minute of examination—the answer to this question will aid the drafter in determining which testing format (e.g. essay, multiple-choice) is best equipped to achieve the drafter’s goal.
Next, Professor James Wollack of the University of Wisconsin-Madison discussed the “Implications of Standardization: Test Security for LSAT and Bar Exam.” More specifically, his talk focused on exam cheating in the academic setting.
Shockingly, 68% of undergraduate students nationwide admitted to cheating at least once during the academic year. Of those who admitted to cheating, 85% believed cheating was essential to their academic success, 90% didn’t believe they would get caught, and, in fact, 95% did not get caught. To put all these numbers in perspective, a website which enables students to purchase original pre-written term papers and projects receives 8,000 unique hits each day.
In the admissions test context, over the last decade (virtually) all the major testing industries (e.g. SAT, ACT, MCAT, GMAT, TOEFL) experienced documented incidents of cheating. For example, one student foolishly placed a craigslist ad seeking a surrogate to take his LSAT exam for him. Additionally, across the licensure credentialing industry, cheating is widespread (e.g. radiologist, school bus drivers, sommeliers, and the list goes on). Regarding the bar exam, a member of the NCBE disclosed that six different jurisdictions reported “testing irregularities” regarding 30 different candidates last year on the bar examination.
Professor Wollack identified five factors that impact whether a person will cheat on an examination: (1) the stakes of the examination, (2) the examinee’s predisposition, (3) the perceived need to cheat, (4) the opportunity to cheat, and (5) the perceived punishment if caught. Specifically, potential cheaters begin by weighing the pros/cons of cheating, asking themselves questions like: “Will an important decision be made with this particular test score? Will I get a benefit for a good score? Will I likely be punished for a bad score?” The examinee will then weigh those answers against his or her own personal moral compass. Generally, society is becoming more and more tolerant of cheating. Third, the student will likely weigh their perception of how well they will perform on the exam relative to the target score without cheating. Essentially, they will ask themselves, “Can I pass without cheating?” He stressed that this step is tied to the student’s own self-assessment or perception and is not based on actual objective indicators of success. Fourth, the student will assess whether there will be an opportunity to cheat. Do the test conditions allow for fraud? Professor Wollack stressed that anything proctors allow in the testing room is an area of vulnerability. In fact, there is an entire industry devoted to cheating enabled clothing (e.g. earbuds designed as earrings, cameras in glasses and shirt buttons). Cell phones, however, are the single biggest threat, especially students who utilize a two-phone system: a decoy phone voluntarily surrendered to the proctor at the start of the test and a secreted cheater phone. Lastly, the student will weigh their perception of what will happen to them if caught against the potential for reward or gain.
The definitive source for test administration guidance, including security standards, is the “Joint Standards for Educational and Psychological Testing” manual. The book explains that the examinee’s behavior is influenced by many environmental and administrative factors. Therefore, examiners are wise to manage these conditions thoughtfully. The manual explains, that when designing a test, developers should adopt a comprehensive approach to test security at all phases of the test, beginning at test development, and continuing through messaging with candidates, delivery, post-exam web monitoring, and statistical detection for indicia of test fraud. When a departure from any protocol occurs, the departure must be meticulously documented to ward against fraud.
In response to an attendee’s question, Professor Wollack advised that it was worthwhile to share large scale grading rubrics and testing outcomes with students (especially for purposes of formative assessment), without fear of jeopardizing the test security. Meanwhile, granular level rubrics should be safeguarded, if the test is going to be re-used again in the future. He concluded his presentation by quoting Julie Andrews, suggesting that amateur [test proctors] practice until they get it right; professionals practice until they can’t get it wrong.”
Immediately before lunch, panelists William Adams (ABA Deputy Managing Director), Gage Kingsbury (psychometrician consultant), and Mark Raymond (psychometrician at the NCBE) discussed ABA Standard 314 and explained “How Formative and Summative Assessments differ from a Standardized Admission Test (LSAT) and a Licensure (Bar Exam).” The panelists outlined the need for various types of assessment and explained the pros and cons of the different types of assessment. Much of what was mentioned in this hour is very familiar to academic support professionals but was undoubtedly helpful to the other folks in the audience. The most popular piece of advice during this hour came from Dr. Raymond. He suggested that to draft quality “distractor” answers to multiple-choice question, the professor should first ask the question as an open-ended response question. Then, on a later test administration, take the most popular wrong answers supplied by the students and use those responses in the newly-converted multiple choice question.
After the lunch break, attendees could choose from four breakout sessions: formative assessment, LSAT scoring, bar exam scoring, or fairness issues in standardized testing.
In the bar exam scoring session, Kellie Early began by describing the three major components of the exam. She then explained how each multiple-choice question undergoes a three-year vetting process, which includes soliciting feedback from recently barred attorneys. (As an aside, the LSAC also uses a three-year timeline for launching new questions on the LSAT.) Meanwhile, essay questions are not vetted in advance of a testing administration because they are too memorable and thus subject to security breaches. All MBE and written questions are selected for inclusion on the examination in the 8 to 18 months before the exam administration. The examination booklets are formally printed about two months prior to the administration and shipped to the local jurisdictions in the final month.
Next, Douglas Ripkey explained the difference between the terms raw score, scaled score, equating (MBE), and scaling (written component). Equating refers to the statistical process of determining comparable scores on different exam forms, while scaling is the process of placing written scores on the same scales as the MBE. To equate the MBE, the NCBE relies upon a subset of previously used questions (a.k.a. equators) to establish a baseline of proficiency over time for each testing group. To scale the written component, the NCBE relies upon the belief that there is a strong correlation between performance on the MBE and performance on the written components. First, one must determine the mean and standard deviation of MBE scores in the testing group. Next, one must determine the mean and standard deviation of the written scores. Lastly, using z-score statistics, the NCBE “rescales the essay scores so that they have a mean and standard deviation that is the same as the MBE mean and standard deviation.” He likened the process to converting Fahrenheit degrees to Celsius degrees, with each measurement system using shared anchor points such as the temperature at which water boils and water freezes. Mr. Ripkey concluded by stating that statistically speaking all applicants could “pass” the bar examination in any testing administration.
In the last hour of the conference, participants could choose from workshops exploring formative assessment techniques, the future of the LSAT, the future of the bar exam, or fairness issues in standardized testing.
In the fairness workshop, Dr. Mark Raymond (psychometrician at the NCBE) and Ben Theis (test developer at LSAC) explained how high-stakes test makers handle fairness concerns. Typically, test developers look for bias in three places (i) at the test question level, (ii) at the test score level, and (iii) at the decision level.
LSAC employs two separate dedicated “fairness” reviews, one by an external committee and one in-house. Once the questions are included on the pre-test section of the LSAT, LSAC tracks the question’s statistical analyses. The developers are using the “pre-test” section of the LSAC to vet these questions with a diverse testing group. Then they look for “residual” differences or “DIF” on a scatter plot. Plainly stated, individuals with the same objective criteria (male, aged Y, with X degree) should both perform equally well on the exam. If they don’t, then the developers look to see if race or gender could account for the statistical difference. LSAC employs a “presume unfairness” mentality for any questionable item. If the item exhibits any unfairness qualities, the item is discarded permanently without further evaluation,
LSAC revealed that minorities are over predicted on the LSAT. In other words—more frequently than is true for other populations—the minority student’s LSAT score suggests that the student will perform better in law school than they do. While more research is needed, the findings suggest that for some reason minority students are under performing in law school despite solid LSAT scores.
To summarize, this one-day conference exposed the numerous attendees (including law school deans, faculty, academic support and bar exam specialists, and admissions personnel) to a solid foundation in the hot topics associated with standardized testing in legal education.
(Kirsha Trychta, Guest Blogger)
Friday, February 8, 2019
In my last post, I argued that banning all external sources of learning, so-called “supplements,” undermines students’ crucial use of self-regulated learning and decreases students’ perceived autonomy support. This, in turn, weakens learning. This time around, I will discuss how blanket bans on external sources deprive students of one of the most powerful tools in a learner’s arsenal: “The Testing Effect.”
Legal education as a whole seems to treat testing as a tool strictly for use in formative and summative assessment. If we want students to “see where they are at” during the semester, we use testing. If we want to give students a grade, we use testing. So conceived, testing serves only to assess and not much else; it serves to evaluate learning and not to create knowledge, per se. In 2006, Roediger & Karpicke noted that although this view is patently wrong, it nonetheless pervades many areas of education.
But scores of studies show that testing is one of the best ways to construct, comprehend, and retain knowledge. Since as far back as 1909, scholars have demonstrated the superiority of testing over “restudying,” and empirical inquiry into this phenomenon has flourished in the last 15 years. For instance, in 2007, McDaniel and colleagues found that both short answer and multiple choice quizzes significantly enhanced learning to a greater degree than additional reading, even though the quizzes did not directly target the exact material tested on the final assessment.
Moreover, in a 2010 study, Buttler concluded that the testing effect enhanced performance even on subsequent tests where “transfer” was necessary. In other words, the assessment test required students to apply their learned knowledge to unfamiliar contexts. This finding is thus particularly relevant to the study of law, where students must learn to analogize the facts of one case to a new fact pattern.
Perhaps the skeptic might say: “Fine. Testing is learning. But, law professors do not have the time to write, grade, and give feedback on so many tests.” Quite true, but I am not advocating for a regime in which law professors provide the testing. The theme of my scholarship is that we should empower students to teach themselves and not let students outsource the responsibility of learning to us. (I would even argue that self-testing is more optimal given its necessary attachment to metacognition and self-regulated learning.) Luckily, it just so happens that the testing effect is a stronger learning tool than re-study even if self-provided and even if it lacks feedback. In other words, even though faculty provision of the testing is beneficial and enhances the testing effect, professors need not be involved in the process necessarily.
That brings us to the topic of banning outside sources of learning. If we want to heed the extensive empirical evidence of the superiority of the testing effect, but we are unable to provide students with the essential resources, blanket bans on outside materials necessarily deprive students of the testing effect. Students are then left with the sole recourse of re-reading notes and outlines, which all available evidence shows is a substantially less effective learning method. We are therefore actively undermining student learning and sending a normative message about how future study (such as bar exam preparation) should be carried out.
As a result of all this, I suggest that faculty point students in the direction of quality self-testing resources while steering them away from sources that lack quality. I will take up the problem of resource quality in my next post which will also discuss counterarguments to my claim that faculty should not enact blanket bans on outside sources of learning.
Friday, January 18, 2019
In my last post, I expanded on my claim that blanket policies against external sources of learning are unwise. I focused on how those policies undermine students’ metacognition by limiting the resources available to eliminate the “unknown unknowns.” In this post, I will continue my critique by focusing on “self-regulated learning” and “autonomy support.”
Self-regulated learning and autonomy support are both positively correlated with successful learning. Therefore, any educational practice that undermines these concepts will also undermine learning and, thus, academic success. What are these important concepts?
Self-regulated learning involves more than just what its name implies. While metacognition focuses on whether a student monitors what they learned, self-regulated learning deals with whether the student monitors and tweaks how they learned. Experts on SRL describe three stages of the process: forethought, performance, and self-reflection. In the first stage, the learner sets goals and considers strategic plans to attain those goals. In the second, the learner sets out on the tasks identified in the first stage and monitors her focus and adherence to the strategies. In the final phase, the learner evaluates whether she met the learning goals, determines the cause of attaining or not attaining the goal, and then restructures her original plan based on this causal attribution so as to improve the learning cycle and provide better results. There is a world of difference between the learner who says “I did poorly on my first midterm because I’m not good at torts” and the learner who says “I did poorly on my first midterm because I studied wrong.”
Autonomy support is a subset of self-determination theory, whose focus is to determine the necessary conditions for optimal motivation and, in turn, optimal performance. SDT’s first precept is that social environment matters in learning. This “social environment” includes actions by instructors. This means nothing more than that instructors can influence students’ success. This can occur when a student perceives a high degree of autonomy support, i.e., an environment in which instructors support the learner’s ownership of learning. Autonomy supportive environments conduce to powerful “intrinsic motivation,” while controlled learning environments conduce to weaker “extrinsic motivation.” As one study put it:
"[A]n individual in a position of authority (e.g., an instructor) takes the other’s (e.g., a student’s) perspective, acknowledges the other’s feelings, and provides the other with pertinent information and opportunities for choice, while minimizing the use of pressures and demands. An autonomy-supportive teacher might, for example, provide students with necessary information while encouraging them to use the information in solving a problem in their own way. In contrast, an authority who is controlling pressures others to behave in particular ways, either through coercive or seductive techniques that generally include implicit or explicit rewards or punishments.” (Black & Deci, 2000)
Importantly, heightened levels of perceived autonomy support are correlated with autonomous self-regulation. Self-regulation, in turn, is correlated with improved academic performance. Therefore, if we undermine perceived autonomy support by using blanket prohibitions against extrinsic materials, we make it less likely that students will self-regulate their learning. If we make it less likely that students will self-regulate, they will not learn optimally. If they do not learn optimally, their performance in law school, the bar exam, and in practice will suffer.
In fact, by overly controlling students’ learning autonomy, we undermine the likelihood that they will be good lawyers. We can all agree that a new lawyer who can act autonomously and without a supervisor’s handholding is a more valuable employee and better lawyer than one who needs constant oversight. By controlling students’ learning methods, we inhibit students’ growth into being “self-regulated lawyers,” and we condemn them to a career of second-rate lawyering and perpetual oversight by superiors.
In my next post, I will discuss how bans on external learning sources run afoul of one of the most powerful tools students can use to improve their learning.
Sunday, December 9, 2018
In my last post, I took up the issue of “blanket policies forbidding supplements.” I argued that such blanket policies squander an opportunity to influence students and that they re-entrench socio-economic hierarchies. In this post, I will continue to contend that such policies are generally unwise, but I will focus now on arguments arising out of the science of learning.
- Killing Metacognition. If Donald Rumsfeld taught us just one thing it is that the “unknown unknowns” are the biggest problems. That is, the biggest problems are the ones we do not even know that we do not know. This is true in learning the law as well. After underperforming on an exam, a student might say “I knew that course backwards and forwards.” The problem, though, is that she only knew what she knew … she was blissfully unaware of the things she did not even know she did not know. How does a student fix this problem when she does not even know what she is missing?
The answer is metacognition. Roughly speaking, metacognition the practice of skeptically monitoring one’s own knowledge, learning, and progress. That skepticism – always pushing back on that “illusion of mastery” – compels the student to explore her learning assumptions and root out the things she did not even know she did not know. If, for instance, she takes a practice problem and gets it wrong due to a doctrinal misunderstanding, she just discovered a misunderstanding that otherwise might have hurt her performance on an exam.
Students need extrinsic sources to support metacognition. If the knowledge they have gained from the traditional sources has left them with (unknown) learning gaps, it is patently illogical to guide them back to those same sources. By imposing a blanket policy against the tools of metacognition (tools like Joe Glannon’s questions in his fantastic E&E series), the professor has just undercut one of the most powerful tools of learning.
But, what about the problem of conflicting sources? Each professor likely has certain nuances that differ from the sea of supplements out there. I would rebut this argument on two grounds. First, this is why we should lead students towards “hornbooks” and not “supplements.” Focusing on hornbooks, i.e., sources written by professors who are experts in their fields, reduces the chance that multiple sources will lead to doctrinal discrepancies. I also lead students away from “supplements,” sources not written by professors, because I have observed doctrinal errors or difference in nuance in these sources.
Second, I think it is key not to let the perfect be the enemy of the very, very, very good. Although slight distinctions might exist between one professor and another, it is entirely rational to believe that the law in required courses is settled to the degree that any doctrinal distinctions between faculty and quality hornbooks will be limited in number and de minimus in scope. The benefits of metacognition are so great that we should not undermine metacognitive practice just because of slight differences in nuance. See generally Preston, et al., Teaching 'Thinking Like a Lawyer': Metacognition and Law Students, 2014 BYU L. Rev. 1053 (2015) (noting the importance of teaching law students the skills of metacognition).
Banning outside sources undermines the crucial skill of metacognition and, in turn, leaves students without these important skills as they become practitioners. They become dependent on the “sage on the stage,” which after law school takes the form of the law firm partner who has little time to lecture to a neophyte lawyer who lacks the skills to find answers herself.
In my next post, I will continue to line up arguments that push back on the practice of banning outside sources.
Wednesday, October 24, 2018
Hello ASP blog readers! I want to thank Prof. Jarmon and the ASP Blog editors for letting me join in the discussion. I look forward to posting some hopefully thought-provoking posts, perhaps challenging some pedagogical orthodoxy along the way. I will be addressing topics mostly focused on the intersection of legal education and the science of learning. For instance, I will be discussing whether the use of Learning Styles Theory is justified, whether cognitive science suggests that we should abolish law school “supplements,” and how ASP and bar prep faculty can mitigate the degree to which certain aspects of legal education re-entrench privilege.
I should also acknowledge my colleague Prof. Raul Ruiz. Prof. Ruiz teaches FIU Law’s bar preparation courses and incorporates in his pedagogy many of the most effective aspects of educational psychology and cognitive science. His teaching has had an enormous impact on our students’ bar exam success, and his helpful critiques have helped me sharpen the ideas that follow.
Again, my thanks to the Law School Academic Support Blog crew.
Sunday, April 9, 2017
Thank you to Kandace J. Kukas, Assistant Dean & Director of Bar Admission Programs at Western New England University School of Law, for providing us with a write-up regarding this recent New York event:
Fifteenth Successful Conference in the Books!
The New York Academic Support Workshop celebrated its fifteenth year at New York Law School on Friday, March 31, 2017. With more than 25 attendees from New York, New England and the East Coast, the one-day conference re-energized ASP professionals for the end of year and bar exam stress, as well as creating thoughtful discussion of how to more effectively and efficiently reach our students.
Hosts Kris Franklin of New York Law School and Linda Feldman of Brooklyn Law School set up a robust agenda consisting of insightful conversations, activities and sharing. One of the day’s highlights was the “Improvising Your Way to Good Legal Analysis” session led by Victoria Eastus of New York Law School. All participants formed a circle and learned how improv can help break the ice with our students as well as show the students they already know how to implement legal analysis. Our everyday living requires analysis, when shifted to the legal arena the connections proved to be quite powerful and generated quite a buzz in the room!
Additional “take-aways” from the day included a recognition that a flipped class may have some draw backs for students who learn differently, project management is a crucial part of legal work and we can guide our students through creative means to understanding task management, and none of us likes homework; especially ASP Professionals! In addition, we discussed and shared experiences creating an Academic Support Program, helping the students with doctrine comprehension using creative methods, and the components necessary to writing well; rules, doctrine, issues and facts. Finally, we shared our time management techniques, with students and ourselves; we are pulled in many directions as are our students, we can use ourselves as guides and examples of superior time management.
Most importantly we shared support and comradery! It is crucial that every now and then we reach out beyond our school’s walls and remember there are a number of brilliant people across the country working very hard to ensure student success in law school and beyond. These friendships and the support felt in the room are some of the reasons I am honored to be in academic support!
Wednesday, December 16, 2015
Our guest post today is by Louis N. Schulze, Jr., Assistant Dean and Professor of Academic Support at Florida International University College of Law. He served on the faculty of Suffolk University Law School (2004-07) and New England Law | Boston (2007-14), earning tenure at the latter in 2012. He is a former Chair of the AALS Section on Academic Support.
Outsourcing Academic Support is a Problematic Proposition
I have been intrigued recently by the discussion occurring on the Academic Support listserv. One member of our community posted a request for information about whether, and to what extent, schools partner with and/ or outsource bar preparation and academic support to bar prep companies. Some schools have dabbled with partnering, and other schools report full-scale immersion. What I took from all of these reports (and my own discussions) was that bar prep companies seek not only to have a hand in for-credit bar prep courses but also in the area of traditional academic support. This troubles me.
I write to express my belief that the wholesale outsourcing of academic support to bar prep companies, though perhaps an attractive proposition for some deans, is a questionable one when viewed through the lens of assessing what is best for our students, our institutions, and the legal profession.
I. Some Preliminary Matters.
First, I think a dichotomy exists between bar prep companies’ role in curricular bar preparation during law school and bar prep companies’ role in academic support during law school. Unsurprisingly, bar prep companies are quite good at bar prep. For reasons explained infra, I believe that bar prep companies are less able to meet the unique goals of academic support.
Part of my thesis relies on this distinction; while I am somewhat more optimistic about partnering with bar prep companies for curricular bar preparation during law school, I am far less sanguine about the increased presence of bar preparation companies in the area of academic support. I think this dichotomy is a crucial one for deans and academic support professionals to digest, as there is a material difference between these two realms.
Second, I think a distinction exists between “partnering” with a bar prep company and the “wholesale outsourcing” of courses or programs. Unlike the bar prep/ academic support dichotomy I posit above, I see this distinction more as a spectrum than a binary choice. The lightest form of “partnering” would likely be adopting a bar prep company’s materials and questions in a course, while the opposite end of the spectrum (the “wholesale outsourcing”) would entail having a bar prep company fully teach and administer some facet of a law school’s offerings.
In my view, as a law school’s choices increase from “partnering” towards “wholesale outsourcing,” those choices become more questionable. While there are no doubt many acceptable points along the spectrum, law schools ought to think carefully about crossing the threshold between partnering and outsourcing, especially in the area of academic support.
II. My Arguments.
My thesis is that law schools should not outsource academic support, per se, to bar prep companies. I am less concerned about partnering/ outsourcing law school bar prep courses. I am even only mildly concerned about partnering with bar companies in the area of academic support. But what I fear is that bar prep companies, in the name of diversifying product lines and increasing profits, will seek to dominate not only the bar prep market and the law school bar prep course market, but also the field of academic support. In my view, such a result would do more harm than good to our students.
But why is this so?
1. First, one-on-one academic support is the most effective academic support, but it is not the most cost-efficient. Anyone who teaches law school academic support has had the experience of watching a student’s eyes light up as they have the big “ah ha!” moment. Some call this “the law school click.” It occurs when a student suddenly makes multiple connections, all at once, and realizes exactly what her professors are getting at – why we use cases to teach law; why creating outlines is important; why we test the way we do; why one must “argue both sides”; why all these methods make students better lawyers.
Usually, this moment occurs in an office with both student and ASP professor huddled over a desk, reviewing an exam, a paper, or some other work product. This moment is usually preceded by other less fruitful in-person moments, but the point is that the “ah ha” moment is one that happens over time and in a one-on-one setting. While it’s true that our ASP classes facilitate these moments, and give the framework and coursework for the moments of enlightenment, I’ve found that the “ah ha” moments happen in-person.
This is much less likely to happen if academic support is provided by bar prep companies. Why? Bar prep companies are corporations, and as such they owe fiduciary duties to their investors. They do not owe fiduciary duties to the students they are teaching. As a result, if they can cut costs by reducing costly endeavors they can and must do so. The first item on the chopping-block would be the costly method of one-on-one, individualized meetings.
2. Academic support is not one size fits all, but one size fits all is cost-efficient. Each law school’s academic support methods differ significantly from the methods of others. This has a lot to do with the differences in administrations, faculties, students, and missions of each law school. Applying the methods of one school to that of another would be ineffective because academic support must be tailored to the environment of the law school. An approach to the contrary waters down the effectiveness of the program, plain and simple.
But, one size fits all is cost-efficient. If a corporation could fashion an academic support program that could be installed as-is into multiple law schools, such a program would increase the profit margin of the endeavor. By contrast, tailoring an academic support program to the unique needs of individual schools (let alone students) would be cost-inefficient. Changing aspects of the curriculum to account for differences in faculties, students, and other stakeholders would require person-hours, and person-hours come with a price tag. As a result, because bar prep companies are corporations, and corporations have a fiduciary duty to the bottom line, academic support would likely become one size fits all.
3. There are many purposes for academic support, but bar passage is the ultimate purpose of any bar prep company. Law schools provide academic support for myriad reasons: to decrease dismissal rates; to support students from traditionally underrepresented backgrounds; to humanize the law school environment; to communicate performance expectations more expressly; to increase bar passage rates; and to make students better lawyers. Each institution may emphasize these purposes differently, but at the very least each of these is likely on the table in terms of justifying expenditures for academic support.
But, the purpose of a bar prep company is solely to promote bar passage. While this purpose might coincide with some of the other purposes, it likely subordinates them to a “lesser” status. Moreover, setting bar passage as the sole or primary purpose of academic support could actually be antithetical to the other goals. For instance, one could argue that to increase bar passage rates, a law school should actually increase its dismissal rates. That way, only the strongest students “count” in terms of bar passage rate. An academic support program focused solely on bar passage, therefore, might spend little time on saving 1Ls and all of its time on those who managed to get through. Although this approach might increase a school’s bar passage rate, it would utterly fly in the face of schools’ duties to the students they admit.
4. Successful academic support relies heavily on an ASP faculty’s engagement with other faculty. If academic support is outsourced to bar prep companies, whose employees would not be embedded in the institution full-time (under the proposal with which I am familiar), the academic support program would lack the crucial element of connection to the institution’s faculty.
This point relates to “buy-in,” and a successful academic support program must have it from both faculty and students. Students buy in to an academic support program if they know that there is a regular and positive collaboration between their doctrinal faculty (who will grade their work) and their academic support faculty. Meanwhile, doctrinal faculty buy in to an academic support program when they know, among other things, that the academic support faculty will help students with more than just passing the bar and that the academic support faculty will not re-teach the doctrine or teach in a way that conflicts with their course. Because a cost-efficient bar prep company academic support provider must pop around between multiple law schools, neither students nor faculty can be assured that the support program will embody the type of multi-stakeholder synergy necessary for success.
5. Another crucial element of successful academic support is knowledge of one's students’ strengths and weaknesses and providing counseling that helps enhance the former and mitigate the latter. This happens over time and requires a great deal of relationship building. The level of trust required to develop these relationships seems unlikely to exist if the academic support provider is not imbedded within the institution.
This point relates to the murky intersection of academic support and counseling. While ASP faculty are (mostly) not trained psychological counselors, a great deal of our most effective work occurs on the personal level. An outsourced academic support program might be able to determine that a student is weak on essays, but a true academic support professional will know WHY the student developed this weakness and how to help work the student toward mastering the problem – both on the academic and personal level. An outsourced academic support program simply will not have time to work on this holistic (but critical) endeavor. In short, an outsourced program teaches students; a true academic support program teaches people.
6. Subtle conflicts of interest. ASP faculty are often called upon to be unofficial advocates for the student body. Because we know our constituency so well, we provide robust input in institutional conversations that could impact students. Because we have certain employment protections (and this is just one reason why ASP professionals should be eligible to earn tenure and long-term contracts), we can advocate for students in ways that outside contractors cannot. Because bar prep companies will likely have their own pecuniary interests in mind, they likely will not advocate for students in the same way as ASP faculty.
For instance, many ASP professionals serve on their law school’s Academic Review Committee or provide data to those committees when they decide whether to readmit dismissed students. Student petitions for readmission often paint the rosiest picture for the students’ readmission, while grades and LSAT scores provide only a limited picture of a student’s potential. ASP professionals who have worked closely with the dismissed students can provide information that paints a clearer and more objective picture of whether a school should take a chance on readmitting dismissed students.
Outsourced academic support programs cannot possibly provide that level of objectivity and nuance. First, it is doubtful that an Academic Review Committee would permit an outside contractor ever to serve on such a committee. But even if the committee accepted data and observations from such a source, how could the committee ever trust that the information is objective when the outside contractor has a vested interest in ensuring that no “borderline” student ever sits for the bar and possibly harm the school’s bar passage rate? Why would an outside contractor ever take such a chance when their future contractual relations rely on bar passage? As a result, law schools lose an opportunity for clearer information about their students when they outsource academic support.
Law schools should not outsource academic support to commercial bar prep companies, a proposal that at least one company is marketing. At many schools, in-house academic support programs provide a genuine and effective source for student support. Partnering with such companies in the area of academic support and even outsourcing curricular bar prep courses might be reasonable, but the wholesale abrogation of a law school’s fiduciary duty to prepare its students for success is deeply problematic. Should law schools follow this slippery slope, they slide one step closer to outsourcing clinical, legal writing, and even doctrinal teaching.
Sunday, November 8, 2015
Our Guest Blogger this week is Distinguished Professor Emeritus, Norman Otto Stockmeyer, who retired last year after teaching at Western Michigan University Cooley Law School since 1977. He also taught as a visiting professor at Mercer University Law School and California Western School of Law. Otto taught principally first-year courses (Contracts, Criminal Law, and Research & Writing) as well as Remedies. He received the top teaching award at Cooley Law three times and was voted National Outstanding Professor by Delta Theta Law Fraternity International.
Multiple-Choice Question Guidelines
Law school professors and academic support professionals should use multiple-choice questions for assessment and testing purposes. After all, our students will have to take and pass a bar exam with a full day of multiple-choice questions. It stands to reason that their chances of passing will be enhanced if they have successfully taken myriad multiple-choice tests in law school.
Going one step further, I submit that our multiple-choice questions should reflect the style and format used on the Multistate Bar Exam. The MBE professionals know more about multiple-choice methodology than we do. And if we want our tests to mirror the MBE, we should adopt the MBE’s question-drafting practices.
The following guidelines are derived from a 2008 article in The Bar Examiner, published by the National Conference of Bar Examiners, and an examination of MBE questions released since the examination was redesigned in 2005.
- Use one question per fact pattern. Do not piggyback multiple questions on a single fact pattern.
- Make fact patterns as concise as possible. Do not include extraneous facts unless fact discrimination is the skill being tested by that particular question.
- Make fact patterns realistic and free of bias. Use genderless characters to the extent possible; otherwise equalize the number of men and women in your questions.
- Identify characters generically, rather than by names or letters. (“A buyer agreed with a seller…” rather than “Able agreed with Baker….”).
- Include all facts in the fact pattern. Answers should not introduce additional facts.
- Provide four answers for every question. More choices add complexity with little appreciable improvement in reliability.
- Avoid compound answers (“A and B, but not C”). (Besides, students hate these.)
- Do not use “all of the above” or “none of the above” answers. (Ditto) Every question should have one, and only one, indisputably correct answer.
- Distribute correct answers randomly. Amateur testers tend too often to place the correct answer in the C or D position. Savvy students pick up on this.
The overall goal of these guidelines is clarity, making sure that we are assessing substantive knowledge and legal reasoning, rather than reading comprehension. Making questions easier to read does not make them any easier to answer. It just makes them better questions.
In conclusion, multiple-choice tests can be a reliable way to evaluate knowledge and analytical skill. And researchers have found that test familiarity improves student performance on standardized tests. So using MBE-style questions can heighten the effectiveness of our tests, as well as enhance the performance of our students.
(Readers interested in Professor Stockmeyer's use of multiple-choice quizzes in a first-year course are invited to read his article on “Using Multiple Choice Quizzes” in the January 2011 issue of The Learning Curve. It is available through SSRN at http://ssrn.com/abstract=1736670.)
Wednesday, April 29, 2015
On April 8th, 2015, law school students, administrators, faculty, academic support educators, and admissions officers along with members of the judiciary and leaders within the Law School Admissions Council congregated in a large hotel conference room within walking distance of the Las Vegas strip and a short bus ride from the UNLV William S. Boyd School of Law. They had a common purpose: to discuss how to work together to better meet the needs of our diverse law students. Dr. Terrell Strayhorn gave the keynote speech, an inspirational start to an energizing and thought-provoking three days. Below are my notes from his keynote speech and some of the themes that I took back to Rhode Island with me from the conference. I also have pasted some links below for those of you who wish to read more about the topics touched on in this blog. I have a lot more to learn, but this conference was a wonderful starting point for me, and a much-appreciated opportunity to deepen my understanding of my own diverse students. Much thanks to Kent Lollis, LSAC’s Executive Director of Diversity Initiatives, Rod Fong, Chair of the LSAC Diversity Retention Conference Planning Group, Professor Nancy Rappaport of UNLV, and many others for their hard work in providing this opportunity for all of us.
Dr. Terrell Strayhorn, a Professor of Higher Education at the Department of Educational Studies within Ohio State’s College of Education and Human Ecology, is also the Director of the Ohio State Center for Higher Education Enterprise (CHEE).
During his keynote address, Dr. Strayhorn spoke about the need for students of color to feel that they “belong” to a community, to feel included. In his book, College Students Sense of Belonging, A Key to Educational Success for All Students, Dr. Strayhorn defines a “sense of belonging” as “a basic human need and motivation, sufficient to influence behavior. [It] refers to students’ perceived social support on campus, a feeling or sensation of connectedness, the experience of mattering or feeling cared about, accepted, respected, valued by, and important to the group (e.g., campus community) or others on campus (e.g., faculty, peers). It’s a cognitive evaluation that typically leads to an affective response or behavior.” According to Dr. Strayhorn, a “sense of belonging” is “relational” in that “members matter to one another and to the group,” and that “each member benefits from the group” and the “group benefits from the contributions of each member.”
This sense of “belonging” is an important factor in a diverse student’s potential for success, more significant than her LSAT score. A “sense of belonging” arises from both “structural” and “curricular” diversity. “Structural” diversity refers to the number of diverse students who are in a class overall & within each individual classroom. Curricular diversity refers to bringing both diverse and non-diverse students together in a meaningful way to discuss their experiences and perspectives. Cross-racial understanding comes from this curricular diversity. Simply having a number of diverse students in the classroom does not, by itself, facilitate inclusion. True inclusion involves interaction among students about their different perspectives and experiences. This “interactional diversity” is what impacts the student body. Many law faculty across the country, however, are unready to have these conversations. (See suggestions below)
If law schools do not bring students together to discuss their diverse experiences, cross-racial understanding and inclusion suffers because understanding and inclusion results from these interactions. A lack of conversations in law school classrooms about diverse perspectives among students is a missed opportunity to provide for a deeper sense of belonging for students of color. Students of color need to feel they belong to the community in which they learn. Curricular diversity engenders a sense of belonging, which, in turn, engenders self-efficacy among students of color.
For these conversations to facilitate understanding and inclusion there must be a sufficient number of students of color in the classroom for them to disagree with one another. The risk of having these conversations with too few students of color in the classroom is that these students feel they have to be the spokespersons for their entire race. In terms of structural diversity, law schools across the country still have a long way to go.
Dr. Strayhorn, and, in fact, every member of the panel on that first day, spoke about the importance of effective pipelines that reach deep into the diverse student community as early as middle school or preschool. In addition, he spoke about mentor programs for diverse students, and the need to enhance these programs by providing more oversight and training to the mentors about how to mentor a student. Mentors should not just meet a student for lunch to periodically “breathe on a student.” Rather, he spoke about three steps to being an effective mentor: 1) believe in the students and set high expectations for the students; 2) build character and invest in the students by providing specific strategies, sharing perspectives, and teaching them tools to achieve; and 3) push them to accomplish more (he called it “intrusive exposure”).
Once students of color decide to attend law school, and must choose which school to attend, they typically will view the law school’s website, but do not typically speak with staff or faculty about the law school. Instead, they choose to speak with people outside the law school, particularly family and friends. In fact, during his research, Dr. Strayhorn heard repeatedly from students of color that they chose to attend law school because they wanted to help their family by attaining a well-paying job to make money to give to their family. This family may include spouses and children, but also parents, brothers, sisters, and grandparents or others. In addition, students of color may feel responsible for financially supporting their families while in law school. They have an aversion to taking out debt.
Feelings of belonging also impacted students’ choice of law school: Meaningful connections with law staff and faculty made a critical difference to students of color. Some sentiments that Dr. Strayhorn consistently heard when he asked students why they had chosen their law school was “it was the only law school where the faculty made time to get to know me,” or the staff had an “honest conversation with me about the strengths and challenges of each law school I had applied to.” They “cared about me.” They “helped me with my application.” “Something about the school felt like a family.” Very few students spoke of the law school’s ranking in U.S. News & World Report or the law school’s reputation. Students also rarely spoke about the alumni placement data, bar passage rates, library holdings.
Dr. Strayhorn’s final comments: Minorities are severely underrepresented in the legal profession. The legal profession should better reflect our society. A diverse workforce will make better decisions. Although some great pipeline programs exist, the critical problem facing law schools and diverse students is the lack of a preschool to undergraduate pipeline.
Kathryn Thompson, Director of Academic Success Program, Roger Williams Law School
Wednesday, March 11, 2015
How Do You Solve a Problem Like “Whatever?”
I am a new ASPer; I joined Valparaiso Law School just a few months ago. I was nervous about getting started in a new field. My nerves were not related to lower bass passage numbers; I have faith those numbers will improve in time. I was nervous because I knew I was going to have to wage war with a self-defeating mindset that is too common among current law students.
That particular mindset is summed up in one word: “whatever.” Oftentimes that word is used in the phrase, “whatever happens, happens.” It is a simple phrase with a loaded message. Yes, I agree, whatever happens does indeed happen; very few would people challenge that assertion for its truth. However, “whatever happens, happens” is a terrible mindset during law school, and especially during your bar exam preparation period.
It is clear that many students are entering law school and bar prep already prepared for the possibility of failing. The “whatever” that happens just might be failure, especially if a student is underprepared and lacking basic study skills. As an ASP professional, I push for students to disallow that possibility. I explain to students that I locked myself in my parent’s garage during bar prep and studied to a point that probably would have even made the formidable Paula Franzese unhappy.
I am not saying that we should preach to students, but we should encourage a different mindset. We can’t allow students to think, “whatever happens, happens,” anymore. We have to motivate our students to adopt a mindset to do whatever it takes to make it happen.
As an ASP professional, I meet with students every day who have joined the “whatever happens” mindset, students who are preparing for the negative, whether it is exam failure or bar failure. I am pushing them to embrace the “whatever it takes” mindset. It is easy to just do “whatever.” It requires bravery and discipline to do whatever it takes (to succeed!)
The motivational solution is not to accept “whatever happens, happens,” but to force your desired outcome by doing absolutely whatever it takes to make it happen. Unfortunately, it is not easy to make this concept stick or to present it in a way that is truly memorable.
However, I always like to remind students of the three distinct types of people taking exams. First, is the person who gives it a try, and when they fail, they can say, “Hey, at least I tried.” Second, is the person who gives it their best shot, and when they fail, they can say, “Hey, at least I gave it my best shot.”
Chad Houston, Valparaiso Law ASP
Sunday, February 24, 2013
Leave Your Point of View at the Fact Pattern Door: Part 2 of 2 (Guest post by Seth Aiken, UMass Law)
In the first installment of this post, I suggested that for some law students, life experience and a strongly held point of view can get in the way of law school success. “Older” students, having lived and worked and experienced a little more than most of their peers can tend to let their own point of view and perceptions about the world interfere with legal reasoning. Rather than seeing the legally significant issues in a fact pattern, they focus on the implausibility of the facts and how unlikely or unfair a scenario seems in the context of their own experience or personal values.
With these students, my strategy is to have them start by adding a phrase to the beginning of the first sentence of every essay question, “On an island that you’ve never been to and where no visitors ever go…(essay question begins). I want them to remember that a fact pattern is a closed universe and that adding facts or injecting personal insights into it will only derail their best efforts.
Then I give my students five steps for looking at a fact pattern and drawing out the legally important issues:
- Call of the Question – Start at the end of the exam and read the call of the question so you understand what you are being asked to do.
- Acts – Rather than trying to spot and analyze whole issues, start instead by reading the fact pattern sentence-by-sentence and highlighting any act or failure to act by a party – anything someone in your fact pattern says, does, or chooses not to do.
- Resist Judgment – You do not have enough information yet to know whether any of these acts give rise to a legally significant issue. Resist making any judgment about whether the act is relevant, worthwhile, good, bad or otherwise because all you know right now, is that somebody said or did something.
- Elements – Assuming you studied and know all the elements of every issue you might be tested on, go to each act and consider if it could be one element of an issue. Remember, don’t skip or overlook an act just because it seems like a little thing. The seriousness or severity of the action doesn’t matter. Whether you think the action would lead to a legal action in real life doesn’t matter. What matters is whether that act in the fact pattern, taken at face value could satisfy one element of something you are being tested on. On the other hand, you don’t want to force an issue that simply isn’t relevant. Some facts ARE there to tempt you into a time-wasting, grade-crushing wild goose chase. In order to stay on target, ask:
a) Is the issue you’re thinking about within the testable universe? (i.e. DO NOT analyze a Criminal Law issue in a Torts exam.)
b) Is this issue relevant to the call of the question? (i.e. DO NOT discuss the rights of B vs. C when the question is asking only about the rights of A vs. B.)
c) Are there other facts that satisfy each of the other necessary elements to make out this issue? DO NOT speculate about other elements based on your common sense or some past experience.
Success vs. Relevance – This is the fifth and final step I ask my students to think about because I want the word “success” to trigger a few different cautionary flags.
The success of the issue: Just because a complaining party has a weak case (weak elements) and is likely to lose doesn’t mean the issue isn’t worth raising. If you can make a good faith, “straight-faced” argument that each of your elements is supported by some fact or facts, it is probably a relevant issue, win or lose. In fact if you can make a good faith argument that MOST of your elements are supported by facts, you should raise the issue. Weak facts or a missing element bear on the success of an issue, but are never a reason to not raise it. Being able to explain to your professor why an issue fails is just as important as being able to show why an issue succeeds.
The successes a student brings into the exam: You are walking into the exam with a point of view based in your life experience. Your successes and accomplishments have equipped you to identify and solve many challenging problems, to relate to people and empathize with their circumstances. HOWEVER – here in this exam, you must leave those successes and accomplishments behind. Relating to the people in your fact pattern and empathizing with their circumstances will distract you from seeing what is relevant and keep you from engaging in effective legal analysis.
Seth-Thomas Aitken, UMass School of Law - Dartmouth
Friday, February 22, 2013
Leave Your Point of View at the Fact Pattern Door: Part 1 of 2 (Guest Post by Seth Aiken, UMass Law)
For some law students, life experience and a strongly held point of view can be immense stumbling blocks to law school success.
I began to think about this last semester working with several students in my 1L class. Relative to the majority of law students, these students were older, which is to say they had lives after undergrad – careers, families, mortgages and other “grown-up” milestones. Each came to law school with a clear point of view, seeing his or her world through a lens of experiences, beliefs and ideals accumulated over years. One student had been a nurse and another was a university librarian. One had struggled with substance abuse and one student, already a working mother of four young children had recently earned her undergraduate degree. When I met these students it was clear that each was rightfully proud of where they had been, or at least what they had overcome to get here. They remained very mindful of and connected to the lessons learned in former lives and seemed hesitant to loosen their grip on those memories for fear of losing themselves in the disorienting new world of law school.
As I worked with these students on ways to approach hypothetical fact patterns, I noticed that many had great difficulty issue-spotting. They focused rather on the implausibility of the fact pattern and how unlikely or unfair a scenario seemed in the context of their own experience or personal values. Most often, talking with a student about why he or she didn’t raise a certain important issue in his or her practice answer, I would find out that the student saw the issue, but chose not to raise it, deciding that in “real life” nobody would seriously go to court over those facts, or that it didn’t make sense to spend time discussing an action that would be obviously unsuccessful. Years of engaging in moral reasoning and practical life decision-making seemed to have handicapped these students’ ability to engage in effective legal analysis.
This challenge posed a difficult conundrum. In order to support my students, I needed to connect with them, earn their trust and demonstrate that I sincerely understood and valued who they were and where they had come from to get here. On the other hand, I had to ask them to look past those valuable former-life lessons and experiences in order to develop the analytical flexibility required to succeed in the law.
So my compromise solution has been to adapt an essay exam strategy that capitalizes on the likelihood my students would focus on the story and the actions of the parties in a fact pattern before recognizing the legally significant issues.
I start with one general instruction: Always, always always add a single phrase to the beginning of the first sentence of every essay question, “On an island that you’ve never been to and where no visitors ever go…(essay question begins). I remind them that fact patterns exist in isolation, as if on an island. No facts can be added and no additional facts are needed. They must also be mindful of the island’s inherent hostility and distrust toward visitors, outside opinions or new perspectives. A student’s point of view and common-sense life lessons, while personally valuable and hard-won, will prove confusing and unwelcome if brought to the island and applied to the facts. With this simple, starting prompt, I hope to remind students, whether they are prone to mix life experience with legal reasoning or not, to keep an objective mind about the fact pattern so that they, in turn, don’t lose the objective of the exam. The additional tools I give students to avoid this pitfall and others will follow in a later post.
Seth-Thomas Aitken, UMass School of Law - Dartmouth
Monday, May 14, 2012
The Two Most Common Bar Exam “Confidence Traps” and How You Can Avoid Them
By Ronald D. Dees
Building and maintaining a healthy confidence level is an important component to overall bar exam preparation. There are typically two “Confidence Traps” that cause examinees to be at risk of performing poorly on the bar exam. Students can either be so paranoid about failing that they lose confidence and literally allow their fears to overwhelm them, or on the other end of the spectrum a student can be too confident and thereby underestimate the level of preparation necessary to be successful. By knowing the traits of students who typically fall victim to these “Confidence Traps,” you can evaluate your bar exam confidence level in order to avoid falling into the traps.
The most common confidence trap is the “paranoia trap.” This student is the one who allows his lack of confidence to overwhelm him. The causes of the lack of confidence can be many things. Perhaps the student performed somewhat poorly in law school, or for some reason just does not feel like he can handle the stress and difficulty of preparing for the bar exam.
The key to success for students who lack confidence for any reason is to develop a study plan designed to build confidence through self-assessment and feedback.
If you find yourself in this category, know that you are not alone and the best way to overcome these feelings is to construct a study plan that will help you build a healthy confidence level over time. Doing so will cause you to reach a point where you will know in your heart and mind that you are ready to perform well on exam day.
Many bar takers lack confidence and almost everyone feels overwhelmed by the volume of material they are presented with during bar preparation. Don’t allow such feelings to defeat you. First of all, you must remind yourself that no one gets every question correct or writes perfectly edited script on the bar exam. Secondly, everyone who has ever taken the bar exam felt like they could have used a few more days or weeks to prepare. No student can possibly know “all” of the law. However, if you prepare diligently, on exam day you should feel that you are as well prepared as anyone else in the room and are probably prepared better than most. That feeling itself is a great confidence booster.
Self-assessment and feedback are the keys to building a healthy level of confidence. Throughout your bar preparation, track your progress as you improve on your MBE test questions and take note as you succeed in memorizing more and more law. Write essays and turn them in to someone in your school’s Bar Services or Academic Support Program for review and feedback. If your school does not have such a program, find a professor who is willing to look at your essays and give you feedback.
Another facet of avoiding the paranoia trap is to make a study plan that you feel confident about. It should be one that you know will work for you based on your past success. Stick to your plan, work hard, and work smart. Identify the things you have done in the past that were not helpful or were detrimental to your studying or study habits, and eliminate those things. Constantly assess your plan and be willing to get rid of things that are not working and do more of the things that are working well for you. Keep in mind that you don’t have to have the best score on the bar exam, you just have to pass. However, you want to prepare like you are trying to earn an A, so that even if you have a bad day, you can still be confident that you are capable of scoring a C and passing the exam.
Not as common, but still worth discussing are those bar takers who fall into the “overconfidence trap.”
The student who may fall into this trap underestimates the difficulty, complexity, and demands of proper bar exam preparation. This student typically self identifies as very intelligent and falls into one of two personality types. She may have been either a top-of-the-class “Law Journal Type,” or she was what I refer to as the “Voluntary Under-Achiever.”
The key to success for both of the above types of students is to develop a healthy respect for the difficulty of the bar exam.
The Law Journal Type thinks to herself, “I am one of the top law students in my class, so passing the bar exam is not going to be a problem for me. I will study for it a little, but I am smart enough to pass the bar exam without much real effort. I mean, after all, it is just a test of minimal competency. No problem for a smart girl like me.”
The Voluntary Under-Achiever has trouble staying devoted to studying and typically does just enough to get by. She is smart and knows it. After all she was smart enough to make it through law school with average or above average grades while putting forth only moderate effort. She thinks to herself, “I am a smart girl and I pretty much skated through law school with no problem, so I can do the same on the bar exam. I’ll put in a few hours of study time here and there and maybe cram a little just before the exam, but I don’t need to study for hours and hours, day after day, week after week. I never had to study that way in college or law school and I still passed, so the bar exam should be no different.”
Both of the above types of students need to develop a healthy respect for the difficulty of the bar exam. If you fit into either of these personality types, you need to realize early on that the bar exam is a lot like law school finals, except that it is about five to ten times as difficult. You see, final exams in law school are usually tested over a two week period and you may even have some time to prepare between tests. Furthermore, you typically take four or five classes per semester, so you only have to prepare for four or five legal subjects. Also, many times one or two classes might be a “paper class” where turning in a paper is the final, and one or two classes might also have an open book exam. So, you actually only have to memorize the law for one to three subjects to prepare for finals.
On the bar exam, you will need to memorize the rules of law for 15 to 20 subject areas depending on what is tested in your state. Thus, it will take five to ten times as much work to prepare for the exam as it did to prepare for finals. If you spent three to four weeks preparing for finals, you would need thirty to forty weeks to prepare for the bar if preparing at the same pace that you used in law school. Now, obviously you don’t have that many weeks between graduation and the bar exam, so you are going to have to devote more time per day and per week to studying for the bar than you did in law school, even though you are really smart. Otherwise, you risk being poorly prepared and having an unsuccessful result on the exam.
Developing a healthy respect for the difficulty of the bar exam will help you avoid falling into the overconfidence trap and will motivate you to develop and stick to a study schedule that reflects the time commitment necessary to properly prepare.
In conclusion, the key to this success for all students is to balance a healthy respect for the difficulty of the bar exam with the confidence that comes from being well prepared on exam day. You can allow that healthy respect to motivate you to prepare properly, and in turn, knowing that you are well prepared will help you maintain your level of confidence, reduce stress, and improve your performance on exam day.
Monday, June 20, 2011
Our Guest Column for today is a posting by Barbara McFarland, Director for the Office of Student Success Initiatives at Chase College of Law, Northern Kentucky University. Barbara has suggested an excellent tip for first-year law students and included an exercise to help them apply it. Thank you for sharing your insight and expertise with all of us, Barbara! (Amy Jarmon)
One More Tip: Remedy Writing Problems
Dr. Amy Jarmon’s May 19th blog post provided ten excellent pieces of advice for incoming students. She is kindly allowing me to add an eleventh: Remedy writing problems before you begin law school.
Even students who have always been good writers struggle to master the intricacies of legal writing. Students who are not good writers do not have time during first semester to learn the basic rules of writing good English prose, punctuating properly, and editing for clarity and concision. While we can say that our students should have mastered the mechanics during undergrad, or even earlier, the sad truth is that many of them have not. They have studiously avoided any class that required them to write anything more than a name on a scantron. Or, if they have done any writing, it was assessed by teachers and professors more interested in commenting on the substance than the form.
When my law school offered a voluntary writing course in the week before classes began last August, almost half of the incoming full-time class attended. The improvements achieved during that one-week class, as measured by pre- and post-tests, were impressive. A second post-test given at the end of the first year of law study indicates that some, but not all, of the gains made during that week were retained nine months later. More number crunching is needed to confirm this initial impression, but the good news is that it’s not too late for our incoming students to learn the rules needed to improve their writing.
How they go about that task is up to them, of course. They could take a business or technical writing class at a local college or university this summer, beg help from the high school English teacher who tried to teach them those rules back in the ninth grade, or just buy a book. Grammar and writing books abound; any used bookstore will have inexpensive texts that will serve the purpose. Online grammar guides are also plentiful.
For a simple technique that students may find helpful, suggest this exercise.
Often, mechanical errors are much easier to find in our own writing after the passage of time. Pull up a document you wrote some time ago, read it critically, and use it to diagnose areas of weakness in your writing.
First, double space after each period and review each sentence in isolation:
- Is each group of words between the capitalized first letter and the end punctuation a complete sentence?
- Do the subject and verb match in number and make sense together?
- Does every verb that requires an object have one?
- Are modifiers close to the words they modify?
- Does every pronoun have an antecedent, and do they match in number?
- Are the sentences typically very long, containing two or three thoughts that could be separated?
- Are the sentences typically very short, dividing ideas that could more effectively be communicated in compound or complex sentences?
- Does the sentence structure vary sufficiently?
- Does every word of each sentence convey the precise meaning intended?
- If you read the sentence aloud with great inflection and pregnant pauses, does the punctuation seem appropriate, necessary, and correct?
If the answer to any of these questions is “no,” chart the errors to identify patterns and problem areas. Once you have identified your errors, learn how to fix them by reading in a grammar book or online service. Rewrite each sentence to fix the sentence-level problems.
Then reunite all the sentences for a particular paragraph and review each paragraph in turn:
- Is the first sentence a topic sentence that accurately portrays the remainder of the paragraph?
- Is every sentence in the paragraph related to the stated topic?
- Do the remaining sentences present ideas or information in a logical order for the purpose of the paragraph?
- Are relationships between sentences clearly made by references and other transitional devices?
- Do the remaining sentences develop the stated topic as completely as needed?
If not, identify, chart, and remedy errors. Rewrite each paragraph into a coherent and correct whole.
When you finish reviewing all of the paragraphs in a particular section of the document, look at the entire section:
- Do transitional devices between the paragraphs develop the overall topic or theme of the section?
- Are the paragraphs in a logical order, facilitating the development and exposition of that topic or theme?
- Are the paragraphs typically overly long, too short, or a good mix of lengths?
- Are one- or two-sentence paragraphs used only sparingly and for emphasis?
Again, identify, chart, and remedy errors. Follow the same procedure with as many written documents as possible until you can identify and eliminate errors accurately and efficiently. If you can write and punctuate good sentences and paragraphs, you are more likely to successfully adapt to the forms and structures of legal writing.
Enjoy the rest of your summer, and I will look forward to meeting you in August. ___________________________________________________________________
Although this exercise was created specifically for students who have not yet started law school, it can be easily modified for use with current law students. Unfortunately, many law students are taken by surprise when we expect them to write perfect English prose. Even those with good mechanics are astonished that their writing style, honed by years of trying to write enough to meet the minimum page requirements of undergraduate papers, must be simplified, clarified, and slashed to meet the expectations of their legal writing professor.
We do our students a service by preparing them for legal writing, in addition to warning them about other rigors and oddities of law study. Recommending that they take time now to remedy writing problems is another step toward the goal of informing and educating our incoming students even before they reach our classrooms.
Thursday, September 3, 2009
When Professors Say Dude: Millennial Aren’t the Only New Kid on the Block by Hillary Burgess Experts like Tracy McGaugh and James Dimitri have provided us with great information about how the Millennial generation is quite different from past generations of students and how we can adjust our teaching to allow them to better serve them as they enter our discourse community and professional community. I have so much respect for Tracy, James, and others who are thinking critically about how to best reach and teach our students. I have to wonder, though, if the struggles we are facing are not just that the students are different, but that we, the professors, are different, too. This thought first occurred to me when I walked out of class and saw a very old Volvo in the faculty parking lot. I remarked out loud to myself, “Dude, check it out!” I then became quite self-conscious. Had anyone heard my remark? What would the Boomer profs think of me saying, “Dude?” Would they forever banish me to the status of Sean Penn’s girlfriend at Ridgemont High? (Which would be anything other than "totally rad.") What would my Millennial students think of me saying “Dude?” Am I that old lady who thinks she’s so cool, but really is the antithesis of cool? (Actually, I really am the antithesis of cool.) Then, I realized that GenX is in that awkward ‘tween phase. After running through a number of “like totally bogus” off-limits expressions that I would “like totally like” never “like ever” use again and musical references that I would have to banish (Hey Mickey, the Bangles, and anything New Kids on the Block) no matter "what a pity," I began thinking about how the culture shock that the legal academy is experiencing might not just be about the students. It might be about the professors, too. In the past decade, the generation Xers have come of age enough that we are now teaching in law schools in significant numbers. In a culture where the Boomers started teaching over forty years ago and even the youngest Boomers have been teaching for twenty years, Xers have become the new kids on the block, at least in the professor world. Could the changes we perceive in our students result, at least in part, from the way Xers and Boomers teach differently (generationally generically speaking, of course)? Are our cultural expectations about how students “should” behave so different that our students are trying to navigate a rather schizophrenic system of rules where what is good in Professor Xer’s class is not tolerated in Professor Boomer’s class (and vice verse)? While this type of experience is good training for succeeding in the practice of law, when we talk about the culture shock that is hitting the legal academy, should we also include ourselves? I leave it to the experts in generational studies to theorize about and answer the questions I raise here. Moving forward, I’d love to see our discussions about how we can best serve our current generation of students expand from the perspective of how different the students are to the perspective of how different we all are now that Xers have left the role of students to join Boomers as professors. Especially as a 'tweener, I'm hoping that we all avoid the us v. them mentality as we explore these groundbreaking pedagogical ideas about how to better serve this generation of students.
When Professors Say Dude: Millennial Aren’t the Only New Kid on the Block
by Hillary Burgess
Experts like Tracy McGaugh and James Dimitri have provided us with great information about how the Millennial generation is quite different from past generations of students and how we can adjust our teaching to allow them to better serve them as they enter our discourse community and professional community. I have so much respect for Tracy, James, and others who are thinking critically about how to best reach and teach our students. I have to wonder, though, if the struggles we are facing are not just that the students are different, but that we, the professors, are different, too.
This thought first occurred to me when I walked out of class and saw a very old Volvo in the faculty parking lot. I remarked out loud to myself, “Dude, check it out!” I then became quite self-conscious.
Had anyone heard my remark? What would the Boomer profs think of me saying, “Dude?” Would they forever banish me to the status of Sean Penn’s girlfriend at Ridgemont High? (Which would be anything other than "totally rad.") What would my Millennial students think of me saying “Dude?” Am I that old lady who thinks she’s so cool, but really is the antithesis of cool? (Actually, I really am the antithesis of cool.) Then, I realized that GenX is in that awkward ‘tween phase.
After running through a number of “like totally bogus” off-limits expressions that I would “like totally like” never “like ever”
use again and musical references that I would have to banish (Hey Mickey, the Bangles, and anything New Kids on the Block) no matter "what a pity," I began thinking about how the culture shock that the legal academy is experiencing might not just be about the students. It might be about the professors, too.
In the past decade, the generation Xers have come of age enough that we are now teaching in law schools in significant numbers. In a culture where the Boomers started teaching over forty years ago and even the youngest Boomers have been teaching for twenty years, Xers have become the new kids on the block, at least in the professor world. Could the changes we perceive in our students result, at least in part, from the way Xers and Boomers teach differently (generationally generically speaking, of course)? Are our cultural expectations about how students “should” behave so different that our students are trying to navigate a rather schizophrenic system of rules where what is good in Professor Xer’s class is not tolerated in Professor Boomer’s class (and vice verse)? While this type of experience is good training for succeeding in the practice of law, when we talk about the culture shock that is hitting the legal academy, should we also include ourselves?
I leave it to the experts in generational studies to theorize about and answer the questions I raise here. Moving forward, I’d love to see our discussions about how we can best serve our current generation of students expand from the perspective of how different the students are to the perspective of how different we all are now that Xers have left the role of students to join Boomers as professors. Especially as a 'tweener, I'm hoping that we all avoid the us v. them mentality as we explore these groundbreaking pedagogical ideas about how to better serve this generation of students.
Monday, April 6, 2009
When preparing my briefing workshop this semester, it occurred to me how hard it was to create a well formatted issue statement. Think about it: one of the most common formats for an issue is: whether [most crucial fact of case] constitutes [crucial element of rule] where [most relevant facts of case]. So the issue statement might be easier to formulate after students understood the facts. But understanding which facts are relevant and which are distracters is hard before students understand the rule and the reasoning. Even the rule is hard to put together in a cohesive, well-articulated format as a first step.
So then it occurred to me that it might be easier for students to brief backward: conclusion (who won), reasons (where they can piece together discreet information), rule, issue, and then facts.
When I proposed this idea to my students, they all looked at me like I had two heads. (Don’t I wish!) But, a few days later, many of these same students popped by my office with light bulbs flashing above their heads, indicating they understood the cases better and faster using this technique.
I more fully explain the logic behind this technique and why it could be easier for novice law students in the Teaching Methods Newsletter, Winter 2008, on page 7.
By Hillary Burgess
Friday, January 16, 2009
As we talk to students after first semester grades come out, I find that often I neglect to address one concern that I believe they all have, yet one that is not often expressed in our conversations with students. We have no trouble focusing on grades, exam writing, study habits, briefing skills, etc. Yet I find at the bottom of many of my students concerns, maybe subconsciously for some, lurk a couple of nagging questions. If I am struggling academically, will I be a good lawyer? Will I be able to make a living practicing law? Embedded in these questions are perhaps the concern about repaying loans and living up to the expectations of others.
Students usually find some comfort when they realize that the correlation between academic performance and the potential for a successful practice career is not as strong as they might imagine. I try to get students to think of the whole process of becoming a lawyer as hurdles to be jumped only once. Once you’ve cleared the hurdles (LSAT, school, bar) then you’re at the finish line ready to practice and nobody really cares, particularly your clients, how difficult you found the hurdles.
I usually tell students some true stories to help them with this concern. We all know of students who struggled academically and then went on to fame and fortune or at least successful practices. I share the stories of some people I know like this. Also, we all know of superior academicians who, because of a lack of other skills, could never make a living as a practicing attorney. In fact, some of these people would have trouble giving away legal service, let alone getting someone to pay them for it. (If you are now thinking of some of the people you know in academia, shame on you!) I practiced for ten years and never once did a client ask me what I made in evidence when deciding whether to hire me for a trial. As an aside, I did hear a story of an assistant district attorney once who cited his performance in evidence class as authority for his argument regarding a piece of evidence. The court was not persuaded.
Students that struggle find some comfort in knowing many stellar legal careers have sprung from less than stellar law school performances. Even if this is not verbalized by the student, I think most of the time they have concerns about their ability to practice and make a living. It is a worry that we can help to alleviate. And after all, every thing that we can help students become comfortable with is likely to take them to a better place, both emotionally and academically.
Russell C. Smith
Assistant Dean for Student Services
Campbell Law School
Buies Creek, NC
Friday, October 10, 2008
Bar Passage Training Lesson: More Experiential Learning
By Hillary Burgess
Hillary.burgess [at] hofstra [dot] edu
I am on my way back from the LSAC Academic Support Bar Passage Programs Topical Workshop. I cannot believe how much I learned, especially about questions that I didnt know to ask. However, the biggest lesson Ive taken away from the workshop had nothing to do with bar support or bar passage.
The biggest lesson for me was that no matter what percentage of experiential learning exercises I incorporate into my Academic Success workshops, I can always include more and talk less. My new teaching mantra is going to be, Stop talking to start teaching. I can apply the same lessons to my skills-building workshops that I apply in my casebook courses: no content is so important that it cant be cut in favor of an exercise that teaches students how to learn the content on their own. This principle is true, even when my content is how to learn. Exercises simply do the job better.
I cant thank this community enough for creating the open, caring, and supportive environment we have, from the incredibly supportive wise (surprisingly young) elders to the people who have been around just long enough to not feel new (at least to the new people like me). Both groups don't seem to be afraid to put it all out there if doing so will better serve our community and especially our students.
Wednesday, September 24, 2008
Director of Academic Support, Sandra Day O'Connor College of Law at Arizona State University
The recent loss of novelist and professor David Foster Wallace was both tragic and eye opening. Wallace was not only a renowned thinker and writer; he was also a fellow educator, albeit in a different area. Wallace famously expressed his feeling that lifelong learning was the only way to pursue a full understanding of the world around us, something he called the “big T-truth.” I had already begun to evaluate my work as a new Academic Support Professor, but his passing pushed me to think more deeply, about the nature of higher learning and the challenges that we face as teachers and about the ways in which our roles as teachers, counselors, and mentors, best serve our students as they strive to find the “big T-truth” with respect to their legal educations.
As a new Academic Support Professor, the challenges I faced were myriad, but the sense of accomplishment and delight inherent in facing and conquering them was unparalleled. In looking back on the past year, I found that the challenges I faced broke out into three major groups, which I will term my faculty, my students, and myself.
But before I faced any of those three, I had to first address a threshold question. I needed to define my purpose, to get at what that “big T-truth” was going to be in the universe of my classroom. To clarify my purpose in my own mind, I asked myself probing questions: What role does my course play in the education of students? What specific ideas do I want them to take home? What universal values will I seek to bolster through the course as a whole? What do I want my students to think and feel about the law? About the process of legal learning?
In tackling these first questions, many possibilities occurred to me. Perhaps Academic Support was intended merely to give students a place to go to ask questions, to feel that someone in their school environment was approachable and could formulate clear solutions. That seemed a noble goal, but it fell short of what I thought Academic Support ought to be, and short of everything that I had read Academic Support was intended to be. Was it then a forum where struggling students could obtain the instruction they needed to keep pace with their classmates? That, too, seemed worthy, surely a large part of the overall goal, but somehow not the entirety of what I wanted to accomplish.
In the end, after much thought, I came to a decision. Academic Support, at least for me, would be a program, a set of classes and individual meetings, in which students would feel supported, would obtain necessary instruction, and would benefit from a peer supported learning environment. My Academic Support classroom would also be a forum where students could try on new, difficult, or otherwise missed ideas without fear of embarrassment, where students could tackle legal thinking and learning from a new angle, where students would make understanding their own learning process part of addressing the larger questions that would ensure them improved classroom performance, and where students could rediscover the joy of learning that, more often than not, had carried them through their academic careers before the law school phase, but had fallen off due to the forced curve, difficulty level, or other concerns.
If I could, I decided, my class would provide all of the practical nuts and bolts learning that ASP-ers needed. In addition, I would try to impart a passion for thinking critically about real world legal issues, and, I hoped, some support for the idea that lawyers and legal thinkers must be, above all, lifelong learners. In my mind, the importance of lifelong learning is what Wallace was addressing when he talked about the deep thinking and continual probing of ideas that help us to get at the “big T-truth.” If there was any big picture idea I wanted my students to have, a relentless passion for new knowledge was it. This passion, I thought, would propel them through the remainder of their law school careers and do much to ensure that they became happy, healthy lawyers beyond the university halls.
My goals after answering the threshold question were then two fold: to help students with practical and immediate learning-related issues, and to ensure that the process of addressing those issues resulted in students who were able to carry those skills beyond the ASP classroom. At first blush, these seemed reasonable goals for a new educator, freshly scrubbed and brimming with enthusiasm. And, as I tackled these and other ideas throughout the course of my first year, I found that, these were indeed workable goals, and, I still believe, the right aspirations for my teaching experience.
However, after addressing these threshold issues, I still needed to contend with the other, less theoretical challenges that would face me as a new Academic Support Instructor. I broke these challenges into three categories: dealing with the academy environment as an Academic Support instructor, reaching difficult students, and managing my expectations with respect to both myself and my charges. In other words, they broke into three clear categories, my faculty, my students, and myself.
Dealing with the academy was the first, and most apparent challenge that I faced outside of the classroom. At my home institution, Academic Support was still extremely new and many of our doctrinal faculty seemed unsure as to what it was I would do with all the class time I had been allotted. I caught sidelong glances from time to time when, at a lunch or before a meeting, I would mention something about my classroom or a conference I was slated to attend. I found that it was my duty to dispel what I call the Magic Wand Theory of ASP that many of our faculty held – the idea that, with the help of a few key texts and exercises, absolutely anyone could turn a struggling student into a strong one, and that ASP was not entitled to a legitimate, important status within the legal academy.
Fortunately for me, I work with a fairly open minded faculty, many of whom have recently come from other institutions where ASP was a priority, all of whom love to talk, listen, and debate. They dialogued with me as I explained teaching methods I had just learned about or described conference presentations by which I had been enlightened. They listened when I told them success stories, about brilliant students who had struggled initially and, with the help of ASP, had demonstrated their capability beyond all doubt. And slowly, conversation by conversation, what at first had seemed daunting, began to be pleasant. Over lunch and through faculty colloquia, I formed relationships and dispelled misconceptions. I found that what I had thought was a complex problem could be solved by the simplest of actions; just talking with our faculty has proven the most effective and straightforward means of clarifying the importance and efficacy of ASP. Today there are few, if any, adherents of the Magic Wand Theory remaining and the result of all this talking is that I enjoy wonderful working relationship with many of the people who I once worried would never embrace ASP.
The challenge that has proved far greater, and the one that, after years of preparation for the job, I had anticipated I was most prepared to face, has been dealing with difficult students. This is true in part because the troubles of these students vary tremendously and each student is a unique, individual, each with his or her own strengths and weaknesses, backgrounds and behaviors. I have been at this for less than three semesters and already I have seen issues that run the gamut.
Ranging from extreme PTSD to deep seated family troubles, from a troublesome lack of boundaries to simple anger at not having performed as expected, these individualized problems have presented my greatest challenges. Dealing with each has been very much an exercise in handling students’ personal and learning-related needs in a case-by-case fashion, often working hard to divorce that student’s idea of him or herself as unfixable, from the problem at hand. It was a revelation when I discovered that a student I had thought was blowing off my class was in fact troubled by serious trouble at home. In tackling these problems, I have found that identification is often half the battle. And in the identification game, I have found nothing more valuable than the students’ trust. The rapport building needed to establish this trust takes time, and I have learned to allow the students to open up to me on their own schedules, to allow them the space to confront their problems individually before we take them on together.
Building rapport with students, convincing faculty that ASP is something to be embraced, and facilitating a classroom environment that stands in line with my specific, daily and long-term, global teaching goals has proven daunting in some ways, but exhilarating in most. And, in addressing these goals, my third greatest challenge emerged. This challenge has been pacing myself and allowing things to take time, instead of trying to do everything for everyone all at once. When I began at ASU, I wanted to do so many things. I wanted to maintain the spectacular program my predecessor had put into place. I wanted to both emulate her and develop my own style both in and out of the classroom. I wanted to expand our program and add an upper level ASP class. I wanted to give a presentation to the faculty. I wanted to meet everyone in our wonderful national ASP community. And I wanted to solve every student problem, whether academic or personal, with precision, perfection, and compassion. Needless to say, I learned very quickly that all of these things, together with preparing for a new slate of classes and establishing roots in a new city, were impossible to do all at once and, after months of sixteen and eighteen hour days, the law of diminishing returns found me at last, rendering me too tired to be of much use to anyone and forcing me acknowledge that I had become one of my own professional challenges.
Today, I use a new and far more effective tactic. I allow things to evolve more naturally. Like building rapport with my students and my faculty, developing my program and my professional self takes time. No matter how I try, these things cannot be rushed if they are to be effective. Today, I am delighted to look back on the past year and recognize how far I’ve come. I spent my first two semesters doing the essential, getting to know the material in my classes, the seminal literature in the field, the culture of my institution, and the needs and personalities of the individual students. This semester I have inaugurated a program for upper level students and I will be attending two exciting conferences, where I will deepen my knowledge and expand my network. Next semester, I hope to begin work on an article. The semester after that, well, who knows? I have learned to trust that, with continued hard work and commitment to helping my students succeed, new goals, and the means with which to meet them, will arise in their own time. For now, I am happy to face the challenges inherent in ASP with the knowledge and foresight I have developed over the past year. I am delighted to work everyday toward helping my students through the law school learning experience, guiding and supporting them as they seek out their own, educational, big T-truths, becoming stronger people, more effective law students and, I hope, in the end, lifelong legal learners.
Monday, August 25, 2008
Assistant Professor of Academic Support
Hofstra Law School
hillary.burgess [at] hofstra [dot] edu