Thursday, February 28, 2019
For those of you that just tackled the bar exam this week, here's a few words of congratulations and a couple of tips as you wait for bar exam results.
First, let me speak to you straight from my heart!
Bravo! Magnificent! Herculean!
Those are just some of the words that come to mind…words that you should be rightly speaking to yourself…because…they are true of you to the core!
But, for most of us right now, we just don’t quite feel super-human about the bar exam. Such accolades of self-talk are, frankly, just difficult to do. Rather, most of us just feel relief – plain and simple relief – that the bar exam is finally over and we have somehow survived.
That’s because very few of us, upon completion of the bar exam, feel like we have passed the bar exam. Most of us just don’t know. So now, the long “waiting” period begins with results not due out for a number of months.
So, here’s the conundrum about the “waiting” period:
Lot’s of well-meaning people will tell you that you have nothing to worry about; that they are sure that you passed the bar exam; and that the bar exam wasn’t that hard…really.
Not that hard?
You know that I passed?
There’s nothing for me to worry about?
Let me give you a concrete real life example...
Like you, I took the bar exam. And, like most of you, I had no idea at all whether I passed the bar exam. I was just so glad that it was finally over. But all of my friends, my legal employer (a judge), my former law professors, and my family kept telling me that I had absolutely nothing to be worried about; that I passed the bar exam; that I worked hard; that they knew that I could do it.
But, they didn’t know something secret about my bar exam experience. They didn’t know about my lunch on the first day of the bar exam.
At the risk of revealing a closely held secret, my first day of the bar exam actually started out on the right foot, so to speak. I was on time for the exam. In fact, I got to the convention center early enough to get a prime parking spot. Moreover, in preparation for my next big break (lunch), I had already cased out the nearest handy-dandy fast food restaurants for grabbing a quick bite to eat before the afternoon portion of the bar exam so that I would not miss the start of the afternoon session of the bar exam.
So, when lunch came, I was so excited to eat that I went straight to Burger King. I really wanted that “crown,” perhaps because I really didn’t understand many of the essay problems from the morning exam. But as I approached Burger King, the line was far out of the door. Impossibly out of the door. And, it didn’t get any better at McDonalds next door. I then faced the same conundrum at Wendy’s and then at Taco Bell.
Finally, I had to face up to cold hard facts.
I could either eat lunch or I could take the afternoon portion of the bar exam. But, I couldn’t do both. The lines were just too long. So, I was about to give up - as I had exhausted all of the local fast food outlets surrounding the convention center - when I luckily caught a glimpse of a possible solution to both lunch and making it back to the bar exam in time for the afternoon session – a liquor store. There was no line. Not a soul. I had the place to myself. So, I ran into the liquor store to grab my bar exam lunch: two Snicker’s bars. With plenty of time to now spare, I then leisurely made my way back to the bar exam on time for the start of the afternoon session.
But, here’s the rub:
All of my friends and family members (and even the judge that I was clerking for throughout the waiting period) were adamant that I had passed the bar exam. They just knew it!
But, they didn’t know that I ate lunch at the liquor store.
So when several months later the bar results were to become publicly available later that day, I went to work for my judge wondering what the judge might do when the truth came out – that I didn’t pass the bar exam because I didn’t pack a lunch to eat at the bar exam.
To be honest, I was completely stick to my stomach. But, I was stuck; I was at work and everyone believed in me. Then, later that morning while still at my work computer, the results came out. My heart raced, but my name just didn’t seem to be listed at all. No Scott Johns. And then, I realized that my official attorney name begins with William. I was looking at the wrong section of the Johns and Johnsons. My name was there! I had passed! I never told the judge my secret about my “snicker bar” lunch. I was just plain relieved that the bar exam “wait” was finally over.
That’s the problem with all of the helpful advice from our friends, employers, law professors, and family members during this waiting period. For all of us (or at least most of us), there was something unusual that happened during our bar exam. It didn’t seem to go perfectly. Quite frankly, we just don’t know if we indeed passed the bar exam.
So, here’s a few suggestions for your time right now with your friends, employers, law professors, and family members.
1. First, just let them know how you are feeling. Be open and frank. Share your thoughts with them along with your hopes and fears.
2. Second, give them a hearty thank you for all of the enriching support, encouragement, and steadfast faithfulness that they have shared with you as walked your way through law school and through this week’s bar exam. Perhaps send them a personal notecard. Or, make a quick phone call of thanks. Regardless of your particular method of communication, reach out to let them know out of the bottom of your heart that their support has been invaluable to you.
3. Finally, celebrate yourself, your achievement, and your true grit....by taking time out - right now - to appreciate the momentous accomplishment of undertaking a legal education, graduating from law school and tackling your bar exam.
You've done something great; something mightily significant! Congratulations to each of you! (Scott Johns).
Wednesday, February 27, 2019
One of my colleagues, a lawyer with impressive credentials and accomplishments, claims that one simple practice helped him move from mediocrity to excellence in law school. "I started reviewing my notes every day after class."
What??? It can't be that simple. Wouldn't it be better to read a hornbook -- or two or three -- to thoroughly understand the nuances of the subject? And to follow up with a few law review articles, integrating their insights into the notes that you spend all Saturday retyping? If something intervenes this Saturday to interrupt this deep dive into doctrine, you're sure to catch up next Saturday, or certainly by the next weekend. Except that, for most of us, life intervenes continually. We rarely, if ever, have the time to to accomplish our over-ambitious goals. So that makes it important to follow the KISS principle my engineer father taught me ("Keep it simple, stupid"). As Steven Foster noted earlier this month in his post, Compounding Effect in Law School, small, easy-to-implement practices can yield impressive results over time. So I'm a big fan of reviewing class notes using the technique of "transforming notes" that Dennis Tonsing sets out in 1000 Days to the Bar.
Here's my approach, which adds some tweaks to the method Tonsing describes.
First, commit to reviewing your notes for each class every day before you leave the law building. Making the commitment to staying in the law building to do the review reinforces how important class review is -- in essence, it makes the review a short continuation of class. In addition, it's a great practice to review all today's classes before starting to prepare for the next day's classes.
Second, commit to spending 5-10 minutes for every class you attended each day. That is, you will never spend less than 5 minutes per class, and you will never spend more than 10 minutes per class. Why no more than 10 minutes per class? Because if you stretch out the process of reviewing notes, it will become a chore: the perfectionism monster will rear its ugly head, you'll try to make things perfect, then you won't be able to review the notes before you have to pick up the kids or the dry cleaning, and you'll fall off the wagon. Consistency is more important here than perfectionism. In addition, this keeps the total time devoted to daily review manageable on even the heaviest of class days.
I like using 3X5 or 4X6 notecards for transforming notes. With one card for each class, you immediately get a visual confirmation of how much you're learning, instead of your class review being buried in a thick, over-burdened notebook. Notecards force you to handwrite, which is time-effective and engages the visual, oral, and kinesthetic senses. They are wonderfully tactile and great for kinesthetic learners. Going through a stack of them is an easy way of incorporating spaced repetition into one's study. Perhaps most importantly, the limited space helps reinforce the limited time and limited scope of the review. There's room for fun, too: many of my students have used different colored cards for each class, or use stickers to reward themselves for a job well done.
Now, set your timer (using your favorite technology) and start.
Spend about 2 minutes reviewing your raw notes from class. If you've left gaps, fill them in; if you've written something wrong, correct it.
Now spend 2-3 minutes summarizing your class on the front of your notecard. On the top, write your header: the class date and a sketch outline of where you are in the subject. This helps make sure that you're always putting things into perspective instead of just learning random rules in a vacuum. So, for example, I might write "3/2 -- Crim/Defenses/Excuses/Duress" or "11/17 -- K/Remedies/Legal/Reliance." This visual header also helps later as you build outlines.
Use the rest of the front of the notecard to summarize the most important things you learned in class, including terms of arts, important policies, rules, and exceptions. Think of this as your elevator speech or preparing to answer the question of friends or family, "What did you learn in school today?"
Five minutes to go, and you still have lots to accomplish. Flip over the card -- you'll use the whole back. Draw a line down the middle, leaving a little space on the top. In your remaining time, you'll be asking yourself 3 questions:
How does this connect to something I already know?
What don't I understand?
What ambiguous fact scenario could the professor use to test my understanding of this material?
First, it's time to strengthen your neurological connections to this new knowledge by making a connection and writing it down -- this takes only a few seconds. Make a conscious effort to think about how what you learned today connects to something you already know -- from a previous class in this subject, from another subject, or from something you've experienced or read about. For example, you might think "Duress reminds me of all those old gangster movies where the crook has his gun in somebody's back;" at the top of the page, you might jot down "James Cagney" or "old gangster movie." Your connection doesn't have to be deep or profound at this point: you're just building extra pathways in your brain to what you learned today.
Next, move on to "What don't I understand?" Give yourself several minutes here: it's common to think at first that everything is perfectly clear, then on deeper reflection to realize you have points of confusion. Jot down these questions now, while class is fresh; chances are you'll forget them later. For now, writing the questions is all you need. You can return to them later to get answers -- from the casebook, from a peer who made a good observation in class, from a secondary source, or from the professor. But you can't get the questions answered later if you don't remember them.
Still two minutes to go! Now tackle what for most students is the hardest part of your class review. "What ambiguous fact scenario could the professor use to test my understanding of this material?" It's tempting to give yourself a pass by sliding into rules: "Oh, she'll want to know if we understand the difference between duress and necessity." But you can best test your understanding of the material and flex your analysis muscles by trying to come up with an actual fact scenario. For example, you might think "Duress requires unlawful and imminent threat of death or serious bodily harm to person or third party that a reasonable person wouldn't be expected to resist. So gunpoint = duress, squirtgun ≠ duress, what about a cast iron skillet? Or what if the bad guy is threatening to post information on Facebook that could put the victim's friend in danger of being shot by a gang?" Your hypo doesn't have to be perfect -- it just requires you to engage the class material to try to figure out nuances and hard cases.
That's it! Think of all you've accomplished in ten minutes: you've put the day's class into context, summarized the most important points, connected it to your existing knowledge or experience, identified your points of confusion, and started problem-solving. That's a lot! The first few days of reviewing like this are deucedly hard, but if you stick with it, you will find this daily review sharpens your mind and helps you understand and use the law you've covered in class.
Monday, February 25, 2019
The bar exam starts for nearly everyone on Tuesday, so I want to start with Good Luck to February takers!! By this time, you should be finished or nearly finished studying. Only in rare cases will studying do anything after this point. You need to rest and relax. You studied 2 ½ long months. Obviously, no one knows what will actually happen on the exam, but you put yourself in the right position to be successful. All you could do up to this point is study hard. Now, believe what you did put you in a position for success!
I have a few tips for the next couple days:
- Get a good night’s sleep. You want to be as mentally fresh as possible tomorrow morning. You don’t want to finally wake up after a few questions because every point counts. Be mentally ready to start on question 1.
- If you have trouble getting your mind going in the morning, take 5 minutes to look at EASY bar review material. This is especially true on the MBE. If on the BARBRI test, you struggled with the first 10 questions, do 2-3 really easy questions in the morning. Don’t grade them, because that is not what matters. You just want your mind working when you get into the exam.
- Gather everything you need for the exam tonight and place it by your front door. Include, pens, pencils, any food or drink allowed in the exam (you may want to keep that in the refrigerator), erasers, pencil sharpeners, etc. You don’t want to rush around tomorrow morning. Eliminate as much stress now as possible.
- Re-read the information from the bar examiners. Know what time you must be at the exam location. Also, know what items are allowed/prohibited. You don’t want to take something into the exam that is prohibited. You also don’t want to miss out on items you could take into the exam.
- Eat good meals. Unless you never do it, eat a decent breakfast. Don’t eat too much artificial sugar because you don’t want to crash in the middle of the test. Plan a good healthy lunch. Don’t overeat because you will get tired during the second half of the exam.
- Relax tomorrow night, and don’t worry about studying. You are going through an exhausting 2-3 days’ worth of testing. You must take every break opportunity you can. Your mind will be exhausted by the time you complete the exam, so conserve enough energy to make it through the entire process. I suggest watching a bad movie that requires absolutely no thinking.
- If you are staying in a hotel, don’t forget to pack all your items the day before you plan on checking out. You don’t want to rush around packing the last morning of the exam. Either have someone check you out or leave your things at the bell desk in the morning.
- Do everything you can tonight to make the next few days as stress free as possible. If you need help or ideas, feel free to contact me.
- Keep your energy up during breaks. After lunch, walk around the exam location to keep your blood flowing. During other breaks, do some stretches. You shouldn’t get too comfortable and relax until the day is finished. You don’t want your mind thinking the day is complete until you finish the exam.
- When you wake up tomorrow morning, tell yourself “I will pass the bar today!” Be assertive and believe it. Confidence alone can get you points on the exam.
Remember, you graduated law school and have the ability to pass the exam. Now, go execute your strategy!
Saturday, February 23, 2019
Friday, February 22, 2019
Students using their electronics to multitask in the classroom has been discussed regularly. Everyone has stories about students surfing the Web, instant messaging, and more during class. Many professors have banned laptops because of the distractions to individual learning and to others.
With the move for more online law courses, multitasking may reach new frontiers. Inside Higher Ed posted this week regarding a recent study on multi-tasking during undergraduate online courses. As we often see, there is dispute as to the methodology of the study and comparability of the groups studied. The post is Online Students Multitask.
Thursday, February 21, 2019
Next week, thousands will be headed to convention centers, etc., to show case the handy-work of their bar preparation efforts for the past two months. In preparation, bar takers have watched weeks of bar review lectures, worked hundreds and even thousands of bar exam problems, and created myriads of study tools, checklists, and flashcards.
Nevertheless, with one weekend to go, most of us feel like we aren't quite ready, like we don't really know enough, with all of the rules - to be honest - tangled and knotted up in a giant mess in our minds.
Yet, let me say this up front. Despite how most of us feel, this weekend is not the time to learn more law. Rather, it's time to reflect on what you've learned, to let it live in you, to give it presence within you. But, how do you do that?
Well, as I heard in a recent talk about medical education, I think we've got something important to learn from the medical schools that just might help with bar prep, too. You see, apparently, despite all of the massive amounts of information available from the learning scientists, the philosophy of training doctors boils down to just three very simple steps: "See it--Do it--Teach it."
Here's what that means for the upcoming bar prep weekend: For the past several months, you've been focused on "seeing it" and "doing it." You've been watching lectures, taking copious notes, reading outlines, and working problems. In short, you've been busily learning by seeing it and doing it.
But, for most of us, despite all of that work, we aren't quite sure (at all!) whether we are ready for the real bar exam because we haven't yet taken the last step necessary for cementing and solidifying our learning; namely, we haven't yet "taught it."
So, that's where this weekend comes in.
Throughout this weekend, grab hold of your notes or study tools or checklists or flashcards, pick out a subject, and teach it to someone. That someone can be real or imaginary; it can be even be your dog Fido. But, just like most teachers, get up out of your seat, out from behind your desk, and take 30 minutes per subject to teach it to that someone, from beginning to the end. Then, run through the next subject, and then the next subject, and then the next subject, etc. Even if you are by yourself, talk it out to teach it; be expressive; vocalize or even dance with it. Make motions with your hands. Use your fingers to indicate the number of elements and wave your arms to indicate the next step in the problem-solving process. Speak with expertise and confidence. And, don't worry about covering it all; rather, stick with just the big topics (the so-called "money ball" rules).
What does this look like in action? Well, here's an example:
"Let's see. Today, I am going to teach you a few handy steps on how to solve any contracts problem in a flash. The first thing to consider is what universe you're in. You see, as an initial consideration, there's the UCC that covers sales of goods (movable objects) while the common law covers all other subjects (like land or service contracts). That's step one. The next step is contract formation. That means that you'll have to figure out if there was mutual assent (offer and acceptance) and consideration. Let's walk through how you'll determine whether something is an offer...."
I remember when I first taught. I was hired at Colorado State University as a graduate teaching assistant to teach two classes of calculus. But, I had a problem; I had just graduated myself. So, I didn't really know if I knew the subject because I hadn't yet tried to teach it to someone. As you can imagine, boy was I ever scared! To be honest, I was petrified. Yet, before walking into class, I took time to talk out about my lesson plan for that very first class meeting. In short, I "pre-taught" my first class before I taught my first class. So, when I walked into the classroom, even though I still didn't quite feel ready (at all) to teach calculus students, I found myself walking in to class no longer as a student but as a teacher. In short, I started teaching. And, in that teaching, I learned the most important lesson about learning, namely, that when we can teach something we know something.
So, as you prepare for success on your bar exam next weekend, focus your work this weekend on teaching each subject to another person, whether imaginary or real. And, in the process, you'll start to see how it all comes to together. Best of luck on your bar exam! (Scott Johns).
February 21, 2019 in Advice, Bar Exam Issues, Bar Exam Preparation, Bar Exams, Encouragement & Inspiration, Exams - Studying, Learning Styles, Stress & Anxiety, Study Tips - General | Permalink | Comments (0)
Wednesday, February 20, 2019
The second semester of law school can be a roller-coaster of emotions. Pulled this way and that by grades that don't match one's initial expectations; by advice coming from lawyers, upper-division students, and popular culture; by competition, whether real or imagined; and by the expectations of friends, colleagues, family, and professors, it is easy to turn from healthy self-critique to unhealthy self-criticism. Even students who avoid the trap of identifying themselves with their GPA can be pitiless self-critics, mentally inflating minor flaws (in anything from their legal writing to their resumes to their social skills to their professional attire) into gross deficiencies that they fear will torpedo their incipient legal career.
It's exhausting and counter-productive to be our own worst enemies. We can break the cycle by learning the difference between self-criticism and self-critique, and by practicing self-compassion.
The difference between criticism and critique is one of viewpoint. Self-criticism is emotional and negative: it involves finding fault with oneself or one's work. In contrast, self-critique is neutral: it involves objectively evaluating one's performance. In my first semester in college, I was lucky enough to have a professor who demonstrated the difference. When Professor David Lagomarsino handed back my first history paper, every paragraph was covered with red ink. The thesis of the paper had logical deficiencies, word choices were suspect, sentences were verbose, paragraphs were loosely organized, and the research did not necessarily lead to the conclusions drawn. Reviewing the detailed and copious comments on the first page, and the second, and the third, I began to suspect I had chosen a college far beyond my intellectual capacity. Then, on the last page, I saw the grade -- an A -- and overall comments that indicated I had written a quality paper. From Professor Lagomarsino's rigorous critique (followed by many similar critiques over the next four years), I learned that my reasoning and writing were generally good, but they could be immeasurably improved by dissociating my ego from my work product. It was the most valuable lesson I learned in college.
Practicing self-compassion is an excellent way to move from self-criticism to self-critique. In essence, self-compassion turns the Golden Rule inside out: we practice treating ourselves with the same kindness and consideration that we would extend to a friend or a colleague. For example, if a friend handed us a draft of their legal resume where the first job description was opaque, we wouldn't respond by saying, "This resume is terrible. You'll never get a job with this." Instead, we might ask, "Would an average lawyer understand what your responsibilities were in this job? Is there any way of redrafting the description so someone without a botany background would understand it?" This would be a compassionate way of critiquing the work product to help our friend create the best possible resume. Likewise, self-compassion allows us to critique our own performance (whether on exams or oral arguments or legal resumes), knowing that we are building up rather than tearing down. By consciously practicing self-compassion, we can quiet the carping voice that bullies and belittles us (and, research shows, actually impedes our ability to achieve our goals) and replace it with positive self-talk that helps us to achieve our goals and maintain a healthy attitude and balance in school, work, and life.
Tuesday, February 19, 2019
Law school is nutritionally disruptive. This was common knowledge at my law school, where my classmates and I joked about having gained 15 pounds while we were getting our JDs. We all felt we understood what had happened. For three years we had chained ourselves to our desks, abandoned physical exercise in favor of mental anguish calisthenics, and frequently resorted to fast food or prepared meals to minimize time spent in the kitchen. Some of us still managed to blow off some steam in a bar from time to time, but otherwise, culinary matters took a back seat to our studies. The resulting excesses -- weight gain, or manic caffeine intake, or bingey sugar highs -- were seen almost as a badge of honor, like pulling an all-nighter to get a memo in on time.
As far as I can tell, things are still the same today. Law students beset with too many tasks and not enough time have to find ways to make time or to soothe stress, and meals and snacks offer convenient opportunities to do so. Not every student makes unhealthy choices, and many of those who do face few ill effects beyond the need for a new wardrobe. But now, watching from the other side of the lectern, I can better see that food issues can have noticeable or even serious impacts on some students' academic performance:
- While gaining weight often seems to be no more than a nuisance, to some students, such changes can be associated with actual effects on mental state, such as decreased stamina or alertness, or negative moods. The weight gain may not be the cause of these changes -- it can sometimes be an effect of lifestyle changes in diet and exercise that can be the source of changes to mental state.
- Sometimes dietary changes specific to certain substances -- such as increased intake of alcohol, caffeine, or sugar -- can have particular effects on behavior or mental state, such poor judgment, fatigue, agitation, or distractibility, that can have negative impacts on critical reading, time management, attention to detail, and other keys to success in law school.
- Sometimes the problem is not so much too much food or the wrong kind of food, but too little food. Students facing shaky finances may find their food budget the easiest thing to cut. Other students may not eat enough food -- or at least not enough healthy food -- because of loss of appetite due to stress. Food deprivation can lead to distraction, disrupt blood sugar levels, and affect memory and attentiveness.
When we work with students, especially one on one, we have opportunities to observe whether some of them are perhaps inordinately affected by dietary issues. In some cases, we may need to enlist the help of others. For example, if financial insecurity is manifesting in a poor diet, a referral to Financial Aid may be appropriate. Encouraging students to seek help from physicians or mental health professionals may also be wise when food issues are leading to serious primary health concerns. But sometimes our students just need a little grounding, a little reminder that they have to take care of themselves while they take care of their studies. A few helpful tips can include:
- Eating smaller meals (or healthy snacks) over the course of the day, rather than pigging out on one big meal at the end of the day after classes are over, can help moderate calorie intake and lessen variations in blood sugar levels.
- Planning ahead for the day or even the week can help to insure steady, healthy eating while minimizing time spent in preparing or obtaining food.
- Buying and carrying around healthier snack alternatives can help forestall binge purchases of high-sugar and high-fat snacks during breaks between classes or study periods.
- Scheduling meals with classmates (for study purposes) or friends and family (to stay connected) can be a good way to make efficient use of the time that you have to spend eating anyway, so that good food doesn't seem so much like an expendable indulgence.
When they are stressed out about studies and papers and exams, taking care of themselves may be the last thing on students' minds. Helping them see how beneficial and easy healthy eating can be may help some students' academic performance.
Monday, February 18, 2019
We all dream of new projects that could progress our programs, improve studying, or change our lives. I think about the article I want to finish, but find allocating time to research difficult. Adding more practice questions or additional study time is difficult. Many people look to others lives and wonder how successful people achieved success. We can model some of our behaviors from successful people to maximize our own potential.
I read an article last week from success.com titled “8 Things Successful People Never Waste Time Doing.” Cynthia Bazin said successful people don’t:
- Get Sucked Into Social Media
- Go Through the Day Without a Plan
- Do Emotionally Draining Activities
- Worry About Things They Can’t Control
- Hang Out With Negative People
- Dwell on Past Mistakes
- Focus on What Other People are Doing
- Put Themselves Last in Priority
She proceeds to quickly discuss how each of these activities can waste time and derail progress towards our goals.
Professors, law students, and attorneys could take this advice to improve productivity. The majority of us probably spend too much time on social media. While those apps have some advantages, the downfall is the amount of time spent using them. 30 minutes less on an app could be another article about a research topic, a practice question, or a response to client concerns. 30 minutes at night could be reading an inspiring book or quality time with family. Consider limiting social media, screen time, or both to improve productivity.
Creating a good plan for the day is something I need to do more. I fall into the trap of trying to solve every problem immediately and divert my attention constantly. A better plan could ensure I get through my research.
Check out the article. Pick one area to save time and the one task to insert into the saved time. Efficiency makes a huge difference in what we accomplish.
Sunday, February 17, 2019
Saturday, February 16, 2019
Takeaways from “Best Practices in High-Stakes Testing” Conference
On February 7–8, 2019, the National Conference of Bar Examiners and the Law School Admission Council co-hosted a conference entitled “Best Practices in High Stakes Testing: What Legal Educators Need to Know.” The conference consisted of a welcome reception on Thursday evening, three plenary presentations on Friday morning, and a choice of breakout sessions on Friday afternoon.
After Friday’s opening remarks—which included a nod to academic support pioneer Paula Lustbader—psychometrician Gage Kingsbury explained the difference between admissions testing and licensure testing. Admissions testing is characterized by its lack of a minimum “passing” score and its need for a high level of reliability across a wide range of performances. (Here, the LSAT.) Meanwhile, a licensure exam typically follows a program of instruction, and includes a pre-identified passing score. In addition, a licensure test must possess a high reliability that the pass/no pass line is properly drawn. (Here, the bar exam.)
Both types of exams should possess not only a strong case for the validity of their use, but also the fairness of their use. Validity, in this context, means that the test is being used for an appropriate purpose. (Validity is not the same thing as reliability; reliability is the likelihood that the examination will produce consistent scores over time for all takers in the normal range of performance). Fairness, however, is more complicated. Most psychometricians agree that a test need not ensure that all subgroups (e.g. Caucasians and Latinos) score the same, but that the test should be equally reliable and equally valid for all subgroups.
Dr. Kingsbury highlighted how the term “high stakes” is frequently misused by the testing community. Almost every test is high stakes for someone; the key is to recognize for whom. Is the test high stakes for the test taker, the testing agency, the educators, the community, or some combination of stakeholders? The more stakeholders who are impacted by the results of the examination, then the higher the stakes. Moreover, a real issue arises when the examination is high stakes for one population, but not for another. For example, a statistical “terminal disaster” occurred with the “No Child Left Behind” testing model because the examination was very high stakes for the educators, but a no-stakes examination for the test taker.
He concluded with a quick tip: When crafting high stakes examinations, drafters should pay attention to how much information is being tested per minute of examination—the answer to this question will aid the drafter in determining which testing format (e.g. essay, multiple-choice) is best equipped to achieve the drafter’s goal.
Next, Professor James Wollack of the University of Wisconsin-Madison discussed the “Implications of Standardization: Test Security for LSAT and Bar Exam.” More specifically, his talk focused on exam cheating in the academic setting.
Shockingly, 68% of undergraduate students nationwide admitted to cheating at least once during the academic year. Of those who admitted to cheating, 85% believed cheating was essential to their academic success, 90% didn’t believe they would get caught, and, in fact, 95% did not get caught. To put all these numbers in perspective, a website which enables students to purchase original pre-written term papers and projects receives 8,000 unique hits each day.
In the admissions test context, over the last decade (virtually) all the major testing industries (e.g. SAT, ACT, MCAT, GMAT, TOEFL) experienced documented incidents of cheating. For example, one student foolishly placed a craigslist ad seeking a surrogate to take his LSAT exam for him. Additionally, across the licensure credentialing industry, cheating is widespread (e.g. radiologist, school bus drivers, sommeliers, and the list goes on). Regarding the bar exam, a member of the NCBE disclosed that six different jurisdictions reported “testing irregularities” regarding 30 different candidates last year on the bar examination.
Professor Wollack identified five factors that impact whether a person will cheat on an examination: (1) the stakes of the examination, (2) the examinee’s predisposition, (3) the perceived need to cheat, (4) the opportunity to cheat, and (5) the perceived punishment if caught. Specifically, potential cheaters begin by weighing the pros/cons of cheating, asking themselves questions like: “Will an important decision be made with this particular test score? Will I get a benefit for a good score? Will I likely be punished for a bad score?” The examinee will then weigh those answers against his or her own personal moral compass. Generally, society is becoming more and more tolerant of cheating. Third, the student will likely weigh their perception of how well they will perform on the exam relative to the target score without cheating. Essentially, they will ask themselves, “Can I pass without cheating?” He stressed that this step is tied to the student’s own self-assessment or perception and is not based on actual objective indicators of success. Fourth, the student will assess whether there will be an opportunity to cheat. Do the test conditions allow for fraud? Professor Wollack stressed that anything proctors allow in the testing room is an area of vulnerability. In fact, there is an entire industry devoted to cheating enabled clothing (e.g. earbuds designed as earrings, cameras in glasses and shirt buttons). Cell phones, however, are the single biggest threat, especially students who utilize a two-phone system: a decoy phone voluntarily surrendered to the proctor at the start of the test and a secreted cheater phone. Lastly, the student will weigh their perception of what will happen to them if caught against the potential for reward or gain.
The definitive source for test administration guidance, including security standards, is the “Joint Standards for Educational and Psychological Testing” manual. The book explains that the examinee’s behavior is influenced by many environmental and administrative factors. Therefore, examiners are wise to manage these conditions thoughtfully. The manual explains, that when designing a test, developers should adopt a comprehensive approach to test security at all phases of the test, beginning at test development, and continuing through messaging with candidates, delivery, post-exam web monitoring, and statistical detection for indicia of test fraud. When a departure from any protocol occurs, the departure must be meticulously documented to ward against fraud.
In response to an attendee’s question, Professor Wollack advised that it was worthwhile to share large scale grading rubrics and testing outcomes with students (especially for purposes of formative assessment), without fear of jeopardizing the test security. Meanwhile, granular level rubrics should be safeguarded, if the test is going to be re-used again in the future. He concluded his presentation by quoting Julie Andrews, suggesting that amateur [test proctors] practice until they get it right; professionals practice until they can’t get it wrong.”
Immediately before lunch, panelists William Adams (ABA Deputy Managing Director), Gage Kingsbury (psychometrician consultant), and Mark Raymond (psychometrician at the NCBE) discussed ABA Standard 314 and explained “How Formative and Summative Assessments differ from a Standardized Admission Test (LSAT) and a Licensure (Bar Exam).” The panelists outlined the need for various types of assessment and explained the pros and cons of the different types of assessment. Much of what was mentioned in this hour is very familiar to academic support professionals but was undoubtedly helpful to the other folks in the audience. The most popular piece of advice during this hour came from Dr. Raymond. He suggested that to draft quality “distractor” answers to multiple-choice question, the professor should first ask the question as an open-ended response question. Then, on a later test administration, take the most popular wrong answers supplied by the students and use those responses in the newly-converted multiple choice question.
After the lunch break, attendees could choose from four breakout sessions: formative assessment, LSAT scoring, bar exam scoring, or fairness issues in standardized testing.
In the bar exam scoring session, Kellie Early began by describing the three major components of the exam. She then explained how each multiple-choice question undergoes a three-year vetting process, which includes soliciting feedback from recently barred attorneys. (As an aside, the LSAC also uses a three-year timeline for launching new questions on the LSAT.) Meanwhile, essay questions are not vetted in advance of a testing administration because they are too memorable and thus subject to security breaches. All MBE and written questions are selected for inclusion on the examination in the 8 to 18 months before the exam administration. The examination booklets are formally printed about two months prior to the administration and shipped to the local jurisdictions in the final month.
Next, Douglas Ripkey explained the difference between the terms raw score, scaled score, equating (MBE), and scaling (written component). Equating refers to the statistical process of determining comparable scores on different exam forms, while scaling is the process of placing written scores on the same scales as the MBE. To equate the MBE, the NCBE relies upon a subset of previously used questions (a.k.a. equators) to establish a baseline of proficiency over time for each testing group. To scale the written component, the NCBE relies upon the belief that there is a strong correlation between performance on the MBE and performance on the written components. First, one must determine the mean and standard deviation of MBE scores in the testing group. Next, one must determine the mean and standard deviation of the written scores. Lastly, using z-score statistics, the NCBE “rescales the essay scores so that they have a mean and standard deviation that is the same as the MBE mean and standard deviation.” He likened the process to converting Fahrenheit degrees to Celsius degrees, with each measurement system using shared anchor points such as the temperature at which water boils and water freezes. Mr. Ripkey concluded by stating that statistically speaking all applicants could “pass” the bar examination in any testing administration.
In the last hour of the conference, participants could choose from workshops exploring formative assessment techniques, the future of the LSAT, the future of the bar exam, or fairness issues in standardized testing.
In the fairness workshop, Dr. Mark Raymond (psychometrician at the NCBE) and Ben Theis (test developer at LSAC) explained how high-stakes test makers handle fairness concerns. Typically, test developers look for bias in three places (i) at the test question level, (ii) at the test score level, and (iii) at the decision level.
LSAC employs two separate dedicated “fairness” reviews, one by an external committee and one in-house. Once the questions are included on the pre-test section of the LSAT, LSAC tracks the question’s statistical analyses. The developers are using the “pre-test” section of the LSAC to vet these questions with a diverse testing group. Then they look for “residual” differences or “DIF” on a scatter plot. Plainly stated, individuals with the same objective criteria (male, aged Y, with X degree) should both perform equally well on the exam. If they don’t, then the developers look to see if race or gender could account for the statistical difference. LSAC employs a “presume unfairness” mentality for any questionable item. If the item exhibits any unfairness qualities, the item is discarded permanently without further evaluation,
LSAC revealed that minorities are over predicted on the LSAT. In other words—more frequently than is true for other populations—the minority student’s LSAT score suggests that the student will perform better in law school than they do. While more research is needed, the findings suggest that for some reason minority students are under performing in law school despite solid LSAT scores.
To summarize, this one-day conference exposed the numerous attendees (including law school deans, faculty, academic support and bar exam specialists, and admissions personnel) to a solid foundation in the hot topics associated with standardized testing in legal education.
(Kirsha Trychta, Guest Blogger)
Friday, February 15, 2019
Assistant Director of Academic Achievement
Position Summary: The Assistant Director supports the Academic Achievement department to train law students for the rigor of law school, the bar exam, and the practice of law with academic workshops, courses, and individual coaching. The Assistant Director will primarily design and implement workshops for students from pre-matriculation through the second year, supervise Academic Fellows, and provide individual tutoring throughout law school and preparation for the bar exam.
The Assistant Director reports to the Director of Academic Achievement and collaborates with the Student Success Team in other law school departments. The Assistant Director is a member of the School of Law staff. Qualified candidates will be eligible for a faculty appointment as an Instructor.
Demonstrate an attitude that reflects the mission and values of the University and School of Law.
- Collaborate with the Director of Academic Achievement to refine, improve, coordinate and direct the program of academic achievement including, but not limited to the summer admissions program, the academic support program, and the bar preparation program.
- Design and implement academic support programming for first-year students, including skills workshops, the academic fellow program, and outreach to at-risk students.
- Collaborates as a member of the Student Success Team to provide programming and support for the overall goal of improving student success.
- Work with the faculty to integrate academic support programming into first-year classes.
- Coordinate and provide tutoring to students in one-on-one and group settings.
- Develop, coordinate, evaluate, and monitor remediation/study plans and activities for individual students.
- Develop, evaluate, and administer the Academic Fellow program, including supervising the academic fellows.
- Coordinate with other departments on student success initiatives, including implementing individual student success plans.
- Teach skills focused courses as assigned by the Associate Dean for Academic Affairs.
- Other duties as assigned
- Must be able to bend, stoop, and reach.
- Manual dexterity to efficiently operate a computer keyboard and other business machines.
- Near vision sufficient to read written communications and computer display screens.
- Adequate hearing to communicate effectively in person and by phone.
- Must use computer keyboard at least four to six hours per day.
- Work is primarily indoors, but requires the incumbent to be in an outdoor environment when traveling between campus buildings or off campus.
- Standard office hours are 8:00 a.m. to 5:00 p.m. Monday through Friday, but evening hours are also required.
- Occasional attendance at weekend events is required.
- Off-campus, state and regional travel is required.
- Incumbent will be exposed to frequent noise caused by telephones and office machines.
License to practice law in one of the United States or the District of Columbia
- Excellent academic record (transcripts will be submitted with application materials)
- Superior written, oral, and interpersonal communication skills (writing sample will be submitted with application materials)
- Excellent organizational skills
- Demonstrated proficiency with technology including MS Office Suite, Internet, common software/applications and the ability to acquire new technology quickly
- Commitment to working with a diverse population of students, faculty and staff
- Sensitivity to students with varied learning styles, disabilities, backgrounds, etc.
- Ability to work under pressure
- Ability to build and maintain a rapport with students
- Skill in presenting information, presentations and delivering instruction
- Ability to collaborate effectively with School of Law faculty and administrators
- Proficiency at project management, planning, and developing goals.
A Juris Doctor degree from an ABA accredited law school is required. A minimum of six months of experience is required. Experience should be in the areas of academic advising, academic support, teaching (adjunct instruction accepted), and/or tutoring within an ABA accredited law school.
A suitable combination of education and experience may be substituted for minimum requirements.
Oklahoma City, Oklahoma, United States
Available at full-time university rates
Position Closing Date
Open until filled
Additional Required Application Materials
List of Professional References, CV or Resume, Cover Letter
Special Instructions to Applicants
Standard office hours are 8:00 a.m. to 5:00 p.m. Monday through Friday; some overtime may be required.
Number of months per year
We have been notified that the link we were given to the Director of Student Success Initiatives position description last week (posting on Saturday, February 9th) was incorrect. The correct link is Director of Student Success Initiatives NKU.
Wednesday, February 13, 2019
Never let it be said that the stuck-in-bed-flat-on-your-back flu doesn't have its consolations. For me, there was the comfort of a kitty snuggling next to me and of the faithful dog taking his post by the bedroom door. There was the sure knowledge that being infectious gave me a temporary pass from committees and meetings, and that my sluggish mental condition made it unthinkable for me to tackle reports, spreadsheets, or even e-mails. But perhaps the greatest consolation was the ability to unabashedly indulge in one of my great pleasures, reading children's books.
I was lucky enough to have close at hand Alison Larkin's splendid audio recording of Arthur Ransome's Swallows and Amazons. For those unfamiliar with this gentle classic, Swallows and Amazons follows the simple adventures of the "Swallows" or four Walker children (and their friends and rivals, the two Blackett "Amazons") as they sail, explore, camp, and meet "the natives" during their summer holiday in the English Lake District. In Chapter 13, the Walker children took an expedition to the west shore of the lake, where they met the local charcoal-burners, "Young Billy" (in his 70s) and "Old Billy" (in his 90s).
For those unfamiliar with this ancient craft, charcoal burning is the practice of carbonizing wood so that it will produce hot, clean-burning charcoal. First, woodcutters would carefully choose the correct fuel, wielding their axes to cut down huge piles of branches, small trees, and undergrowth. Then they would carefully layer these woody materials (taking into account the moisture content, diameter, density, and other characteristics), cover with an airtight layer of earth and moss, and build a controlled, slowly-burning fire which would last for days or weeks. Charcoal burners had to tend their piles night and day -- too little oxygen and the fire would smolder and expire, leaving only charred wood; too much oxygen and the fuel would be entirely consumed, leaving nothing but ashes.
In our novel, when the Walkers arrived at the charcoal-burners' encampment, they spied Young Billy carefully tending his charcoal mound:
A man with a spade was patting the mound and putting a spadeful of earth wherever the smoke showed. Sometimes he climbed on the mound itself to smother a jet of smoke near the top of it. As soon as he closed one hole another jet of smoke would show itself somewhere else. . . .
A big puff of smoke rolled from the burning mound.
"Look there," said Young Billy. "Can't leave him a minute but he's out. . . ." He picked up his spade and went to the mound, where a small tongue of flame was licking a hole from inside. He put a spadeful of earth on the hole and patted it down. . . . "We want ours to burn good and slow," said Young Billy. "If he burns fast he leaves nowt but ash. The slower the fire the better the charcoal."
The charcoal mound is a good metaphor for life. Holes will always pop up, given the vicissitudes of life. We may be on top of our law school studies, but there are dirty clothes in the laundry hamper or we haven't made it to the dentist this year. And as soon as we patch one hole -- for example, we create a good system for doing the laundry as we study criminal law -- another hole pops up: maybe now we haven't changed the oil in the van or we forgot to pick up the dog's medication. How many times have we berated ourselves for not being in complete control of our lives? Prudence suggests it's wise to accept that holes not only can crop up in our lives, but will pop up. Accepting this, we can forgive ourselves for that most human of conditions -- not being perfect.
Moreover, the charcoal-burner Young Billy is as good a mentor as one is likely to find anywhere. If we are hard-working and conscientious, we'll create excellent charcoal mounds (a/k/a outlines/case briefs/appellate briefs/lesson plans/curriculum proposals . . . ). And if our mounds are the result of so much good work, we naturally think we ought to be able to light them on fire and walk away, secure in the knowledge that our efforts will result in excellent charcoal with no further effort on our part. But even the best-laid plans need tending. There is no moment of perfection in which we can be secure and rest on our laurels. Rather, a hole will develop even in the best-created mound, meaning that we will need to pat down a spadeful of earth to make our mound secure -- and this will happen again, and again, and again. In his 70s, Young Billy was the most patient of professionals. However good his mound, he knew holes would develop. Rather than grow frustrated or angry, he paced himself, patting earth against one hole, then patiently patting over the next hole as it developed; in this almost meditative approach, he knew that his patient, consistent practice would over the long haul create consistently good charcoal -- and a good life in which he could be both excellent in his craft and a good neighbor to friends and curious children alike. And that's a pretty good recipe for being a lawyer, too.
Monday, February 11, 2019
If I offered you a choice between 3 million dollars today or a penny today with double the amount each subsequent day for 31 days (ie - .01 today, .02 tomorrow, .04 next days, etc.), which would you take? In a small experiment, over 90% of people took the lump sum. A penny doubled every day over 31 days will net over 10 million dollars. However, that choice doesn’t beat the 3 million lump sum until day 30. Day 31 is when it crushes the lump sum by over 7 million dollars. The reason is the compounding effect.
The compounding effect is a common strategy for financial planning. Starting to save for retirement at age 20 makes a huge difference. Darren Hardy in his book The Compounding Effect applied the theory to life beyond finances to argue all of us can make dramatic changes in our careers and how we live. He argues the small choices we make consistently are the foundation for dramatically changing our life.
Small choices seem trivial. One of my favorite examples in his book relates to weight gain/loss. He illustrates the choices of 3 hypothetical people. 1 person continues to do the same thing he has always done. He is the constant. 1 person decreases calories by only 125 calories a day. 125 calories is the equivalent of a bowl of some cereal or a can of Dr. Pepper. The last person increases calories by 125 calories a day. The small decisions didn’t seem to make much difference in the first 6-12 months. Only small variations in weight. However, from month 12-18, the first person is losing weight and the last person is gaining significant weight. By the end of 2 years, the first person loses 30 pounds while the last person gains 30 pounds. There is a 60 pound difference based on 1 coke a day!
The weight example and other examples in the book illustrate the difficulty of applying the compounding effect to our lives. Choosing to not drink a coke today will not change the scale tomorrow. Eating 1 donut or a meal at Chik-fil-a will not change the scale tomorrow. We don’t perceive the benefits or consequences of small choices immediately, so many people choose the more fun donut. Over time, 125 calories can snowball from weight to other areas of life, like health problems and relationships with friends and family. The good news is the good choices snowball as well, and we should start considering what small changes to make in law school.
Law school epitomizes the problem of no immediate positive reinforcement. Reviewing course material today doesn’t produce a grade. Doing practice questions each week won’t have an immediate impact. Playing the Xbox or going out with friends provides the immediate dopamine rush without the perception of consequences.
Small changes now can make an impact at the end of this semester and future semesters. I want to provide numerous examples and tips for improvement. However, just as my pastor said yesterday, if I give you 4 things to do, you will do none of them. If I give you 1, then you may do it because you can easily integrate 1 thing into your routine. Hardy encourages the same thing. Start small. Pick 1 small thing to improve learning.
My suggestion is a small review each night. Learning literature indicates we immediately begin forgetting material. We can improve retention with immediate review within 24 hours and periodic review throughout a semester at progressively longer intervals. Each night, review the material from that day and the previous day. Summarize the information into your own words and think about where it fits in the outline. Don’t spend numerous hours on this task. Start small and move up.
Compounding makes theoretical sense. However, many of us make small choices that are compounding negatively. Choose something small today to compound in the right direction. Remember, you will not see immediately results. Consistency is what will produce the outcome you desire.
Sunday, February 10, 2019
Call for Proposals
AALS Section on Academic Support
January 2020 Annual Meeting in Washington, DC
"Access to the Legal Profession as a Pillar of Democracy: Bar Exam Cut Scores and the Future of Diversity."
As a community, we are aware that bar passage rates are falling in some jurisdictions, and the ABA recently proposed a change Standard 316 on Bar Passage, which would require schools to achieve 75% bar pass rate within two years of graduation. While this proposal did not pass, the current passage rates still have the potential to severely impact law schools that prioritize a mission of diversifying the legal field by preparing lawyers from underrepresented groups. Moreover, in the profession itself, access to legal education, and the profession, is a pillar of democracy. This program will focus on how to better support students who are at risk of not passing the bar exam.
Topics might include, but are not limited to: student engagement on the bar exam, the impact of the UBE on cut scores, programming to support diverse students in their law school courses, programming to support diverse students while preparing for the bar exam, the role of stereotype threat on the bar exam, and the impact of the changing ABA rules on a diverse student population. Proposals should reflect presentations that will be 25 minutes in length.
Proposals should contain a detailed explanation of both the substance of the presentation and the methods to be employed. Individuals as well as groups are invited to propose topics. The Committee would prefer to highlight talent across a spectrum of law schools and disciplines and is especially interested in new and innovative ideas. Please share this call with colleagues—both within and outside of the legal academy and the academic support community.
Proposals must include the following information:
1. A title for your presentation.
2. A brief description of the objectives or outcomes of your presentation.
3. A brief description of how your presentation will support your stated objectives or outcomes.
4. A detailed description of both the substantive content and the techniques to be employed, if any, to engage the audience.
5. Whether you plan to distribute handouts, use PowerPoint, or employ other technology.
6. A list of the conferences at which you have presented within the last three years, such as AALS, AASE, national or regional ASP or writing conferences, or other academic
conferences. (The Committee is interested in this information because we wish to select and showcase seasoned, as well as fresh, talent.)
7. Your school affiliation, title, courses taught, and contact information (please include email address and telephone number).
8. Any other information you think will help the Committee appreciate the value your presentation will provide.
Proposals will be reviewed on a rolling basis, so please send yours as soon as possible, but no later than Wednesday, May 1st at 5pm to Melissa Hale, Loyola University Chicago School of Law, mhal[email protected]. If you have any questions, please email Melissa Hale.
The Section on Academic Support Program Committee:
Melissa Hale, Chair
ASP Section Chair: Jennifer Carr
Saturday, February 9, 2019
Director of Student Success Initiatives
Northern Kentucky University Salmon P. Chase College of Law
Northern Kentucky University Salmon P. Chase College of Law invites applications for the position of Director of Student Success Initiatives, with an anticipated start date of July 1, 2019. This is a full-time, non-tenure-track position, and it covers both academic support and bar preparation.
The Director’s primary responsibility will be to work with students to help them develop the skills needed to reach their full academic potential for performance in law school and on the bar exam. Responsibilities include: (1) designing and implementing academic support programs; (2) designing and assisting with the law school’s bar exam preparation programs; (3) possibly teaching the law school’s third-year bar preparation course each semester; and (4) possibly teaching the two-semester Legal Studies course for NKU Chase’s evening students. The Director will also supervise the Assistant Director of Student Success Initiatives. Candidates must be willing to work some evenings.
Applicants must have a J.D., a strong academic record, excellent organizational and interpersonal skills, the ability to work collaboratively, and excellent communication skills. Experience in a law school academic support program or other relevant teaching experience is preferred. Candidates should also have an understanding of the dynamics of diversity (including first-generation college graduates) in higher education and a demonstrated ability to incorporate this understanding, as appropriate, in all levels of their work. Salary will be commensurate with experience, but we anticipate the salary being between $75,000 and $80,000 (plus full benefits) per 12-month year.
Founded in 1893, the Salmon P. Chase College of Law has for 125 years provided service to students and the legal community throughout the Commonwealth of Kentucky and across the Ohio River to Cincinnati and southwest Ohio. Long known as the “The Lawyer’s School,” Chase has served both traditional and non-traditional students throughout its history. Chase is fully approved by the ABA and is a member of the AALS. More information about Chase may be found at https://chaselaw.nku.edu/ Application materials (cover letter, CV, law school transcript, and a list of three professional references) should be submitted through the Northern Kentucky University website (https://jobs.nku.edu/postings/7333).
Any questions can be addressed to Lawrence D. Rosenthal, Associate Dean for Academics, at [email protected]. The application period will remain open until the position is filled. The committee will start reviewing applications on February 28.
It is Northern Kentucky University’s policy to ensure equal employment opportunity for all persons and to take the necessary actions needed to recruit, employ, train, promote, and retain qualified faculty and staff, including members of protected groups. Discrimination against any individual based upon protected status, which is defined as age, color, disability, gender, national origin, race, religion, sexual orientation, or genetic or veteran status, is prohibited.
Friday, February 8, 2019
In my last post, I argued that banning all external sources of learning, so-called “supplements,” undermines students’ crucial use of self-regulated learning and decreases students’ perceived autonomy support. This, in turn, weakens learning. This time around, I will discuss how blanket bans on external sources deprive students of one of the most powerful tools in a learner’s arsenal: “The Testing Effect.”
Legal education as a whole seems to treat testing as a tool strictly for use in formative and summative assessment. If we want students to “see where they are at” during the semester, we use testing. If we want to give students a grade, we use testing. So conceived, testing serves only to assess and not much else; it serves to evaluate learning and not to create knowledge, per se. In 2006, Roediger & Karpicke noted that although this view is patently wrong, it nonetheless pervades many areas of education.
But scores of studies show that testing is one of the best ways to construct, comprehend, and retain knowledge. Since as far back as 1909, scholars have demonstrated the superiority of testing over “restudying,” and empirical inquiry into this phenomenon has flourished in the last 15 years. For instance, in 2007, McDaniel and colleagues found that both short answer and multiple choice quizzes significantly enhanced learning to a greater degree than additional reading, even though the quizzes did not directly target the exact material tested on the final assessment.
Moreover, in a 2010 study, Buttler concluded that the testing effect enhanced performance even on subsequent tests where “transfer” was necessary. In other words, the assessment test required students to apply their learned knowledge to unfamiliar contexts. This finding is thus particularly relevant to the study of law, where students must learn to analogize the facts of one case to a new fact pattern.
Perhaps the skeptic might say: “Fine. Testing is learning. But, law professors do not have the time to write, grade, and give feedback on so many tests.” Quite true, but I am not advocating for a regime in which law professors provide the testing. The theme of my scholarship is that we should empower students to teach themselves and not let students outsource the responsibility of learning to us. (I would even argue that self-testing is more optimal given its necessary attachment to metacognition and self-regulated learning.) Luckily, it just so happens that the testing effect is a stronger learning tool than re-study even if self-provided and even if it lacks feedback. In other words, even though faculty provision of the testing is beneficial and enhances the testing effect, professors need not be involved in the process necessarily.
That brings us to the topic of banning outside sources of learning. If we want to heed the extensive empirical evidence of the superiority of the testing effect, but we are unable to provide students with the essential resources, blanket bans on outside materials necessarily deprive students of the testing effect. Students are then left with the sole recourse of re-reading notes and outlines, which all available evidence shows is a substantially less effective learning method. We are therefore actively undermining student learning and sending a normative message about how future study (such as bar exam preparation) should be carried out.
As a result of all this, I suggest that faculty point students in the direction of quality self-testing resources while steering them away from sources that lack quality. I will take up the problem of resource quality in my next post which will also discuss counterarguments to my claim that faculty should not enact blanket bans on outside sources of learning.
Thursday, February 7, 2019
Recently, I heard a discussion suggesting that bar passers do things differently in the final two weeks than those who are not successful on the bar exam. That got me thinking about what I've been seeing, at least anecdotally, in my 10-plus years working with students in preparing for their bar exams.
First, both groups tend to work extraordinarily hard in the last two weeks before their bar exams. So, what's the difference? It must be in the type of work that the two groups are doing. In short, during the final two weeks, it seems to me that bar passers tend to ramp up their practice with lots and lots of MBE questions and essays [while also creating super-short compact homespun study tools (2-3-page outlines, flashcards, or posters)]. In contrast, people who find themselves unsuccessful tend to focus on creating extra-bulky study tools and trying to memorize those study tools with very little continued practice of MBE questions and essays. In brief, one group is continuing to practice for the exam and the other group is focused on memorizing for the exam.
But, here's the rub:
It’s a perfectly natural feeling during the final two weeks of bar prep to want to focus solely (or mostly) on creating perfect study tools and trying to perfectly memorize all the law.
But, according to the educational psychologists, there’s something called “useful forgetfulness.” You see, when we jam packet our study tools with everything, we aren’t learning much of anything because we haven’t had to make any hard decisions about what to let go (what to “forget”). We’re just typing or handwriting or flowcharting like a scribe. But, when we purposefully decide that we are only going to make a super-short “starter” study tools (knowing that we can always add more rules as we work through more questions during the next couple of weeks), our decisions about what to put in our super-short study tools (and what to leave out) means that we actually empower ourselves to know both what we put in our study tools (and what we left out).
As a suggestion, tackle two subjects per day – one subject that is essay-only and one subject tested on both the essay and the MBE exam. Starting with one subject in the morning, using the most compact outline that your commercial course provides (and referencing the table of contents for each subject), create a super-short study tool with the goal of completing your study tool in 2 hours or less.
Here’s a tip:
If you think that you need a rule, don’t put it in because you can always add more later. Instead, only add a rule that you’ve seen countless times over and over. Just get it done. Move quickly. Don’t get stuck with definitions of elements, etc. Stick with the big picture umbrella rules. Think BIG picture. For example, be determined to get through all of contracts in 2 hours (from what law governs to remedies). As a suggestion, have just one rule for each item in the table of contents for your commercial bar review outline. Don't go deep sea diving. Stay on the surface. Then, in the remainder of the morning, work with your study tool through a handful of practice essays. In the afternoon, repeat the same tasks using a different subject (creating a snappy study tool and working through a few essays). Finally, in the evening, work through mixed sets of MBE questions.
In the last week before the bar exam, with most of your starter study tools completed, focus on talking through your study tool (for about one hour or so) and then working through lots and lots essay problems and MBE questions. As you practice in the last week, feel free to add rules that come up in practice essays and MBE questions to your study tool. As I heard one person explain it, your study tool becomes sort of a "bar diary" of your adventurous travels through essays and MBE questions (thanks Prof. Micah Yarbrough!). In short, you've created a study tool that has been time-tested and polished through the hard knock experiences of working and learning through lots of bar exam hypothetical problems.
So, for those of you taking the February 2019 bar exam, focus on practice first and foremost because you aren't going to be tested on your study tool. Rather, you're going to be testing on whether you can use your study tool to solve hypothetical problems. And, good luck on your bar exam! (Scott Johns).
P.S. For those taking the Uniform Bar Exam, there are 12 subjects as grouped by the bar examiners (I think there are 14 subjects in California, depending on how you count subjects):
* Business Associations (Corporations, Agency, Partnership, and LLC)
* Secured Transactions
* Federal Civil Procedure
* Family Law
* Wills & Trusts
* Conflicts of Law
* Constitutional Law
* Criminal Law & Procedure
Tuesday, February 5, 2019
Watching the Super Bowl on Sunday made me think of my students. By no means am I implying any similarity between football and the legal profession. After all, one is a grueling slugfest, featuring breathlessly intense clashes between aggressive competitors on behalf of highly partisan, sometimes even fanatically tribal, stakeholders, with pride and sometimes lots of money at stake. The other is just an athletic contest.
No, the reason I thought of my students was the sudden shift in the game in the fourth quarter. Through the first 50 minutes or so of play, the scores remained unusually low and very close. Neither team could gain a clear advantage, and with less than ten minutes remaining, the score was still only 3 to 3. Across the country, spectators were complaining about how stagnant and boring the game seemed.
Suddenly, within a minute or so of game time, everything changed. The Patriots called a play in which a receiver wound up open in the middle of the field, just beyond the Rams' defensive line, and Patriots' quarterback Tom Brady deftly passed the ball to him and gained several yards. By itself, the play was only mildly exciting, and then only because it provided comparatively more action than most of the preceding gameplay. But the Patriots realized right away that the Rams' defense, which up until that time had been pretty successful at confining the Patriots' forward motion, had not had anything in place to keep that lone receiver from appearing unguarded behind the defensive line. So for the next few plays, the Patriots ran essentially the same play, except for choosing a different receiver to run up the field each time. In the space of four plays, the team moved all the way down the field and set up the winning touchdown run.
From the Rams' point of view, everything was fine, until suddenly it wasn't. They were basically evenly matched with their opponents, in a game that looked like it might go into overtime -- and then, in less than a minute, everything fell apart. One weakness was exposed -- one play that they had no planned defense for -- and before they could adjust to it, the other side had taken advantage of that weakness and taken a significant lead. And the fact that they were in that distressing position left the team vulnerable to more trouble. They rushed and ran riskier plays, hoping to score their own tying touchdown in the short time they had left, and under this pressure the Rams' quarterback Jared Goff accidentally threw the ball into the hands of a Patriots defender, leading the Patriots to score an additional three points. The Rams went from having an even chance of winning to having no chance, all because of one weakness that wasn't addressed quickly enough.
In his fascinating book How We Die, the physician Sherwin B. Nuland explained that human death often follows a similar trajectory. Laypeople often imagine that those suffering from serious injury or illness usually experience a long but steady decline until they pass away. However, Nuland pointed out that at least as often, if not more so, the afflicted person manages the illness (or injury, if it is not so catastrophic that it simply kills them right away) fairly handily, with only slight decline or sometimes even with improvement, for days or even weeks. If nothing bad happens, then they might even make a full recovery. But if one thing goes wrong and isn't corrected quickly enough, it can cause significant damage, which itself leads to additional life-threatening complications, and in a short time the patient may spiral down past the point where any medical intervention will be enough to save him. An infected wound, for example, if not treated quickly enough, can lead to a generalized blood infection, which can cause a patient's kidneys, liver, and other organs to stop working properly, and the patient, who otherwise might have almost completely recovered from his initial injury, will die of multiple organ failure.
We see the same phenomenon in other realms than just sports and medicine, such as business and politics. In any complicated system, there can be long, steady declines, but the sudden drastic reversal, attributable to one or a small number of neglected infirmities, is often more likely.
And the life of a law student is pretty complicated. New information to learn, new ways to think about it, new tasks to perform, all while juggling stress and ambition and self-doubt and mountains of practicalities like housing and relationships and (painfully often) finances. We all know that a few students struggle right from the start, but very often students will be managing -- holding their own, even if not excelling -- and then they run into one tribulation they can't fix, and they can't handle. A course they can't wrap their head around. A romantic breakup. Lack of funds to buy textbooks. A death in the family. An extracurricular activity that takes up too much time.
It almost doesn't matter what the problem is, because it's just the trigger. It starts the landslide that could pull the student down. Struggling in one course, for example, could pull the student's attention away from his other courses, leading to anxiety about not maintaining his GPA . . . and what started as one problem spirals into multiple problems.
The response, from an Academic Success perspective, has to be twofold. First, we need to be able to detect these kinds of issues as early as possible, before they turn into the equivalent of a touchdown by the other team or a raging blood infection. We need to have direct interaction with the students most at risk (incoming students, first-generation students, those in danger of financial difficulty, etc.), so we get to know them and encourage them to be forthcoming. We also need to develop strong networks among those in the faculty and student services who might pass along observations of possible distress.
Second, we need to have systems in place to help these students address these issues quickly, before they do become intractable. We are expected, of course, to handle purely academic issues on a moment's notice. But we should also be familiar with other means of support on campus and in the community, to be able to quickly refer students who need help in financial, psychological, spiritual, and other realms.
Time sometimes really is of the essence. None of us want to end up being Monday-morning quarterbacks, lamenting that if we had just changed our defense one play sooner, we could have saved the game.