Law School Academic Support Blog

July 9th Statement by AASE regarding NextGen

AASE released a statement this week regarding NextGen's haphazard rollout.  You can view the statement here.  I also copied it below.

(Steven Foster)

Scattershot Rollout of The Nextgen Bar Exam May Exacerbate Disparate Bar Pass Outcomes Along Socioeconomic and Racial Lines

Law school graduates will not be sufficiently prepared for the new bar exam scheduled to be administered in 2026. Twenty jurisdictions have already adopted the NextGen bar exam and six of the 20 have committed to administering the exam in 2026 even though crucial information about costs, scoring, portability, and the substantive content is not yet available.[1] Only fractional information has been released about the exam, and there have been far too many substantial changes to that already fragmented information to allow academic support faculty and commercial bar review providers to effectively prepare current law students who will be in the first wave of NextGen examinees.

Amy Meyers, Director of Academic Skills & Bar Success at Willamette University College of Law, reached out to the Association of Academic Support Educators desperately seeking help to fill in the many information gaps about the NextGen exam. In an email to the Association’s president, Meyers pleaded:

“My knowledge is based solely on the NCBE press releases and what I can gather from the [academic support listserv]. As a lawyer, I don’t like relying on hearsay. As an educator, I don’t like trying to teach based on the breadcrumbs I can piece together from those sources. It seems to me that my students should be preparing for such a high-stakes test as the NextGen Bar Exam with more clarity than I can provide. That’s not a comfortable feeling.”

The proprietor of the NextGen exam, the National Conference of Bar Examiners (“NCBE”), has published only piecemeal information distributed over sporadic intervals about the exam’s substantive content and format. Recent statements by the NCBE have contradicted the entity’s earlier promises about the scaled-down exam. The incomplete and inconsistent messaging from the NCBE could potentially impede the performance of bar takers and further contribute to disparate bar passage rates along socio-economic and racial lines. This scattershot approach to communicating essential information leaves law school academic support faculty without the clear, consistent, and reliable guidance necessary to prepare graduates for the new bar exam. With the added emphasis on bar passage in accreditation standards and law school rankings, legal educators cannot afford to silently acquiesce to the significant uncertainties presented by this rollout of an exam that will determine the future careers of law school graduates and the standing of law schools.

After careful scrutiny of all released information, we find the small set of sample questions covering only limited subject matter, coupled with the unclear description of the most recent changes to the not-yet-released exam, wholly insufficient to offer fair and reasonable notice as to the format, content, scoring, scaling, and resources available for the licensure exam. Our specific concerns are summarized below:

 Contradictory Information Regarding Exam Scope and Content

• On May 25, 2023, the NCBE announced the eight subject areas to be tested on the NextGen exam: Business Associations; Civil Procedure; Constitutional Law; Contracts; Criminal Law and Criminal Procedure; Evidence; Real Property, and Torts.
• The NCBE stated, that the new exam will no longer require examinees to have a base of knowledge in the areas of conflict of laws, family law, trusts, and estates, or secured transactions, but these topics may still be included in certain legal scenarios for which examinees are provided relevant reference materials . . . .[2]
• However, on May 29, 2024, the NCBE contradictorily announced that from July 2026 through February 2028, family law and trusts and estates will appear on every NextGen exam in a performance task and may also be included in integrated question sets.[3]

In addition to the late release of the significant additions of two areas of black letter law, we are deeply concerned that the NCBE has not provided a content scope outline for Trusts or Estates. The test maker has stated that these content outlines will be published by “late 2024,” but that illusive commitment still leaves law schools with less than three full semesters to make the necessary adjustments to curricular content.

The information currently provided by the NCBE is too vague to allow prospective NextGen examinees to thoroughly prepare for the exam. Due to these uncertainties, we urge the NCBE to limit the subject matter of the exam to exclude Family Law, Trusts, and Estates. Bar applicants taking an exam in 2026 should have the benefit of knowing the scope of exam content when they entered law school. To do otherwise will place early NextGen applicants at a comparative disadvantage to applicants who will be taking the UBE in 2026 and 2027.

Insufficient Pilot Testing

 The NCBE conducted pilot testing for the NextGen exam from August 2022 to April 2023, but only limited and self-reported information is published about the outcomes of the pilot testing.[4]

Because the pilot testing took place before the decision to expand the scope of the NextGen exam to include Family Law, Trusts, and Estates, we are concerned about the reliability of the pilot test as a predictor for the timing and/or performance on the NextGen exam. We urge the NCBE to conduct additional pilot testing to assure all stakeholders that there will be stability between scores for the current Uniform Bar Exam and the NextGen bar exam.

Legal Reference Materials are Essential to Assess Law Practice Skills

• On June 3, 2024, the NCBE announced that it will not provide legal reference material on the NextGen exam; but that it would provide relevant, targeted resources for some question types, and questions on certain topics within the foundational concepts and principles.
• Nestled in the release of the pilot testing results, is the NCBE’s decision to exclude additional legal resources from the NextGen exam.

According to the NCBE, the aim of NextGen exam is to make the bar exam more realistic and to reduce the amount of legal knowledge that candidates must commit to memory for the exam, while emphasizing law practice skills. Lawyers in practice do not rely on memorization, but on skilled access to legal and information resources. Yet, despite the reality of law practice and its own claimed intention, the NCBE appears to have done an about-face on providing the Federal Rules of Evidence as reference material on the exam.[5] This significant departure does not support the assessment of practice-ready lawyers.

We are reasonably reluctant to rely on the NCBE’s promises of forthcoming materials and the provision of resources during testing because of the many previous changes and the recalcitrant decision to exclude promised reference material that a majority of pilot test takers relied upon. More in-depth research should be conducted by independent psychometricians unaffiliated with NCBE to assess how access to reference materials might impact performance indicators that are not captured in exam scoring like test anxiety and imposter syndrome.

The NCBE Has Offered No Guidance for the Testing of Legal Research

• The NCBE’s website says that it is still “exploring options for testing legal research” and that sample questions testing legal research will be available at a later date that remains uncertain.

The overwhelming majority of current law students who will be taking a bar exam in 2026 and 2027 have or will have completed the required courses in Legal Research and Writing prior to sample questions being released. The unfairness of the promised late release will be compounded on individuals who lack the time and money resources to engage in costly bar preparation programs. Those individuals are more likely to benefit from law school academic support programs, and yet the academic support faculty have not been able to offer any meaningful guidance that will make their bar exam success more likely. We urge the NCBE to delay testing of legal research until 2028 or some future date when law students can have fair notice and access to the scope and content of exam questions in this areas to be tested.

Academic support faculty cannot appropriately aid in the bar readiness of our students and future graduates with vague, incomplete, and constantly changing information from the test maker. We implore the NCBE for its cooperation and recognition of the potential to exacerbate alarming disparities in test outcomes for law school graduates who have the greatest need for academic support programs.

ON BEHALF OF THE ASSOCIATION OF ACADEMIC SUPPORT EDUCATORS[6]

[1] https://www.ncbex.org/exams/nextgen

[2] https://www.ncbex.org/news-resources/ncbe-publishes-content-scope-nextgen-bar-exam

[3] See https://www.ncbex.org/news-resources/illinois-administer-nextgen-bar-exam-2028 (During this period, family law concepts will be tested with the provision of legal resources. Starting in July 2028, family law and trusts and estates will be included in the foundational concepts and principles tested on the NextGen bar exam and will be tested in the same manner as the other foundational concepts and principles.)

[4] https://nextgenbarexam.ncbex.org/reports/research-brief-pilot-testing/#whatis1

[5] https://thebarexaminer.ncbex.org/article/winter-2022-2023/the-next-generation-winter-22/ (“Another major change underway for the new exam is the plan to provide examinees with supplemental materials where appropriate, such as relevant portions of the Federal Rules of Evidence, so that they need to rely less on specific recall of legal doctrine details.”)

[6] The Association of Academic Support Educators is a non-profit professional organization for law school academic support educators. Our members collaborate to develop research-based teaching methods and enhancement programs that empower students to succeed in law school, on the bar exam, and in the practice of law.

 

 

 

1000

 

 

More things change, the more they stay the same

The NCBE released results of the NextGen pilot testing last week.  You can read their report here.

Amazing ASPers throughout the country already started the discussion within our Listserv and on conference calls.  I don't want to reiterate (and/or plagiarize) their wonderful insights.  If some of my arguments sound familiar to someone else's, I sincerely apologize.  I will also provide my shameless plug for joining the bar advocacy committee.  Our complaints are being heard by bar associations, so you are making a difference.

My initial thought turned into the title to this post.  NextGen was supposed to be revolutionary and assess what lawyers do on a daily basis.  The test purported to require less memorization and focus more on practical skills.  The vision was grand, but the result seems to be a lackluster assessment in a slightly different format.

First, and probably most important, the report does not provide transparent data for performance differences among groups of participants.  The report vaguely states that the test narrowed the performance gap with some of the new questions.  The report does not fully say how much narrowing or which groups saw improved scores (it gave a few small examples).  The report also doesn't indicate the baseline for determining "examinee competency".  If the NCBE still uses prior standardized tests (LSAT mainly) to create the baseline for what performance difference is acceptable, then even small narrowing makes the new test seem better.  Most of us argue LSAT/standardized test differences are not based on competency.  Without knowing their baseline of what gaps are expected, I don't trust their argument the new test doesn't continue patterns of discrimination.  Slightly less discrimination isn't persuasive.  I want to be completely wrong here, but the impact is too great to not demand more transparency. 

Second, they aren't following their goal.  Attorneys don't memorize mountains of rules, so proponents of NextGen said it would move away from memorization.  I will concede that a slight decrease happened (no Secured Transactions, yay!).  However, NextGen continually adds back more material to memorize.  Family Law is back on the exam.  The starred vs. non-starred debate rages where we all know students will need to deeply know/memorize material even though the outline says to have a general understanding.  General understanding is too vague to create a good plan, and any reasonable person would over-prepare for the bar exam.  Wills, Trusts, and Estates keeps inching closer to fully back on the exam.  The report says that providing the Federal Rules of Evidence didn't help students/they didn't use them.  Of course students who already studied or engaged in some form of studying (either in bar prep or a for-credit bar class) didn't use the rules.  They already committed them to memory, or possibly, the stakes weren't high enough for them to pull up the rules because missing the questions had no impact on individually passing the exam.  

My last statement is about technology.  I believe the report makes clear the NCBE cares more about efficiency than true assessment.  The report discussed technological efficiency and ability to administer on computer.  The report (and all pilot tests to my knowledge) ignore the cognitive requirements of a 100% online exam (with multiple page PTs).  Why didn't the NCBE give a large number of participants the test on paper and the rest on computer?  My guess, because the answer would be clear.  Hard copies are better for active reading, notes, etc.  Students would most likely perform better with paper exams.

I sincerely hope I am wrong about my assessment of NextGen.  I hope it is a completely different and fair assessment.  My fear is that we are getting a different format of a test that will continue to determine who can reach their dream of practicing law based on screen reading, memorization, fast toggling, standardized testing, and a slew of other irrelevant skills.

(Steven Foster)

Animal Farm (and the 11th Annual AASE Conference).

I am just now coming down from the high of attending the 11^th Annual AASE Conference last week, hosted --in the most gracious way-- by the University of Idaho College of Law in Boise. As one might expect, it was an incredibly collegial and informative academic conference. I know I have mentioned how academic support folks are by the far the kindest, most generous, and scholarly academicians. It is refreshing to be in room after room with colleagues who just get “it.”   “It” is how to get students to: attend workshops, participate in bar review, pay attention to how they learn, and create processes that are efficient and effective. But “it” also is: how to get doctrinal faculty to: attend workshops, keep an eye on students who are falling behind, keep an eye on students who may learn differently. “It” is finally the collective[1] search for information about the NextGen bar exam and whether your jurisdiction will adopt it (and if so, when).

One of the social events at the conference was a dinner and drinks at the Zoo. I mean, who doesn’t love a Zoo?[2] There were giraffes, zebras, lions, and a red panda who I think should have their own Instagram account. It was Tuesday, so there were tacos. It was a different and fun way to build community and enjoy a lovely evening in Boise.

However, I have to wonder if our monkeying[3] around trying to get a handle on a test that we do not know a lot about-including whether it will actually be a thing in many of our jurisdictions- is just caging up the academic support predator that would love to set its sights on this prey by getting to know it (and actually figuring out if knowing it is necessary).

We spent a lot of time talking about the NextGen bar exam together. There were wonderful sessions about how to get ready, or adapt, or prepare students for the NextGen bar exam. In fact, every presentation about bar readiness (and some that were not) discussed the NextGen bar exam. One example was a great presentation about how to use Professional Responsibility as a golden ticket for teaching skills for both the UBE and NextGen bar exams. Another talked about teaching IRAC from a NextGen perspective. We had amazing presentations about how to use AI to Draft Next Gen practice questions, how to prepare students for the new legal research components of the NextGen exam, creating rubrics and learning objectives based on NextGen foundational skills, and more.

But what if after all this careful planning and preparing for NextGen, we, like Columbus, do not find ourselves in the destination we had planned to reach? What if the unknowns we are so carefully trying to infer[4] have been calculated incorrectly? We may already have, or are about to welcome, students who could be taking this bar exam and we have been left with penumbras and emanations.  We have become like Academic Support Ninja Warriors[5] who are trying to get to the goal without knowing exactly what challenges lie ahead. Some of this is the NCBE not being entirely forthcoming about the exam, and another piece of it resides in jurisdictions who have not shown their hands either. Together, that leaves us no better off than the animals in the Zoo: we are at their mercy in terms of getting fed the information we need, and then we wonder if all our hard work to know more than we have been currently told is going to end up being merely entertaining -- but not freeing in any way.

A very special thank you to Karen Wellman and her amazing team at U. of Idaho and the AASE Executive and Programming teams for making this another memorable conference. We are lucky to have each other on this quest. AASE will be posting all the materials and slides from the conference-look in your emails and on the website for more information.

And finally, I am crazy excited that all these amazing people will come to me next year. We will see you in Boston for the 12^th Annual AASE Conference-Suffolk University Law School, May 2025!

(Liz Stillman)

 

[1] But not universal, some jurisdictions have committed with clear timelines in place.

[2] Yes, the animals there may not love it, but bear (get it?) with me for a moment.

[3] The puns will be fast and furious-please feel free to groan-or just shoot me an email that says, “UGH!,” I deserve it.

[4] And asking ChatGPT to help us infer as well.

[5] Which will be the name of my second ASP themed band.

If You Failed the February 2024 Bar Exam…

As results trickle in from the February administration of the bar exam, there are moments of joy and sorrow happening around the nation. If you passed the exam, CONGRATULATIONS! If you failed the exam, please know:

You are not a failure. You are a successful human being.

You know this. I’m just reminding you of the facts.

There are people you can lean on.

It can feel hard to share the disappointment, stress, and even shame you might be feeling. There are people in your life who can’t believe you did something as hard as graduating from law school. There are people in your life who look up to you – who think you are the smartest person they know. There are people in your life who are so proud of you, and they are even more proud of you now for trying. Let them in.

Your school is behind you.

With all the national attention on the bar exam and pass rates, it can feel like you are a statistic. You are not. I graduated from a big undergraduate institution and a large law school. As a student, it felt comfortable to blend into a crowd. I didn’t understand how my faculty members felt about their law school community. Now, as a faculty member, I can tell you, a student’s success is my joy, and their disappointment is my sorrow. I deeply care about my students passing the bar because I care about my students. And I have yet to meet a colleague who doesn’t share this sentiment. Ask for our help and support. We want to give it.

Your February results do not dictate how you will do in July.

It takes over 400 hours of studying, thousands of multiple-choice questions, hundreds of essays, and multiple, timed MPTs to pass the UBE. It also takes stress management. If there were things outside of your control that prevented you from hitting these targets, acknowledge that. Failing the bar exam doesn’t come down to a lack of intelligence or ability; it comes down to dedicating the right amount of time to the right activities for ten weeks.

The work you did studying this past winter is not a waste. It will only ease the burden as you prepare for July.

You will feel surprised at how much you remember when you crack open those books open again.

You are in good company.

Check out this list from JD Advising of impressive people who failed the bar exam.

Try again.

No one will ever ask how many times you took the bar exam, or what you scored. Even if you feel defeated, try again. Being an attorney will define your life. Taking the bar again will not.

(Ashley Cetnar)

To Take a Bar Exam (or Not)

A lot has changed since I started working with law students in 2015. A lot has changed in the world, in my own life, and for the students I serve. It seems every year more students arrive at law school fresh from undergraduate commencement ceremonies. Fewer students have worked in a legal setting, or any professional setting, before embarking on the study of law. This isn’t a bad thing, just a difference I notice with each passing year.

Every August at orientation, the 1Ls are told that they are embarking on a professional journey – that law school is a professional school. We do our best to let them know the expectations are different here than in their undergraduate programs. The workload is heavier. They should be developing professional identities – preparing to one day take an oath of attorney and pledge to ethically represent clients. These are important points that for many describe the path ahead.

Still, less than two weeks out from the bar exam, as the stress mounts on our graduates, I reflect on another change I’ve witnessed over the years. There are more law students who are unsure about whether they want to practice law at all. There are also more law students who are sure they don’t want to practice law. JD advantage career opportunities continue to increase, and students pursue legal education for different reasons than they did a decade ago. Yet the vast majority of graduates continue to take the bar exam.

I’m reminded of the term, emerging adulthood coined by Jeffrey Jensen Arnett, PhD back in 2000. He used this term to describe the period of development between the ages of 18 and 29 where adolescents begin having adult experiences that shape their transition into adulthood. It’s a time of self-exploration and identity development, and this individual growth and personal identity formation is happening alongside many law students’ professional development. I would argue that a growing portion of law students leave law school having made major strides in personal identity formation, while remaining ambivalent about their professional identity.

When I talk with graduates taking the bar exam because of family pressure, because it’s what everyone else is doing, or with no real reason in mind, I know it will be a struggle. Not because they can’t do it, but because they are doing it for the wrong reasons. No amount of external pressure will keep you internally motivated to study forty to fifty hours a week for ten weeks.

Every year, I ask my 3L students to write down why they want to take the bar exam. I ask them to save the piece of paper and look at it when bar prep gets hard. Maybe I should also stop and let them know it’s OK if they don’t want to practice law. There are many successful JDs out there who are happily doing something else with their legal education. I could normalize other paths. I could stop reinforcing the notion that the bar exam is a necessary hurdle for all, and instead be clear that it is only necessary for those who seek to practice.

(Ashley Cetnar)

Oregon Approves Alternative Pathway to Licensure

Attorney licensure may have just experienced a massive shift in practice, and the shift was not from NextGen.  Oregon approved an alternative pathway to licensure that could provide the framework for other states to shift away from standardized testing.  You can read the brief release here.

Debate will ensue regarding the ability of this exam to measure competence.  Whether it works or not, I applaud Oregon for attempting to find a way to measure competence without using bubble sheets.  I also applaud everyone who worked on the proposal, lobbied for change (some within our community), and continue to advocate both for students and minimum competence.  Keep up the good work.

(Steven Foster)

 

Statement from AASE Concerning the NextGen Bar Exam

STATEMENT FROM THE ASSOCIATION OF ACADEMIC SUPPORT EDUCATORS

CONCERNING THE NEXTGEN BAR EXAM

The Association of Academic Support Educators (AASE) has serious concerns about the prototype questions released by the National Conference of Bar Examiners (NCBE) for the NextGen bar exam scheduled to be administered in July 2026.

The NCBE’s Testing Task Force, in their final report released in April 2021, recommended less emphasis on memorized material and greater focus on lawyering skills to more reflect the practice of law. NextGen purportedly tests applicants on skills they actually need to be successful attorneys.  Unfortunately, the recently released exam structure and fourteen (14) questions do not fulfill that promise. 

Significant memorization will be required on the NextGen bar exam.  The NCBE outline displays some topics in each subject with a star and some without a star. The legend explaining the meaning of the star versus no star topics clearly shows that everything will need to be memorized.  “Topics without a star symbol – Topics without a star symbol may be tested with or without provision of legal resources. When these topics are tested without legal resources, the examinee is expected to rely on recalled knowledge and understanding that will enable the examinee to demonstrate recognition that the topic is at issue in the fact scenario.”  Since the language indicates non-starred areas may require memorized knowledge, applicants must memorize everything.

The July 11, 2023, and August 18, 2023, releases create additional uncertainty regarding the exam.  In the July release, the multiple-choice section of NextGen Bar was described as “Initially, many of these questions will closely resemble Multistate Bar Examination (MBE) questions; this will ensure stability between scores for the current and NextGen bar exams. In future administrations, the variety of multiple-choice question types will increase.”

The statement raises a significant concern.  Graduates will be preparing for an exam that is quite literally a moving target.  The NCBE provided no information about how the “variety of multiple-choice question types will increase.”  They only provided 14 questions to represent countless rules and skills.  Graduates and law schools do not know what that variety looks like, how significant is the increase in variety, and how it will impact studying.  In the August press release, the exam structure once again changed from previous announcements clearly illustrating the moving target.  For a high-stakes licensure exam, a moving target with so few examples released in advance is inappropriate.  Graduates have the right to know the exact make-up and nature of the exam they will take and have access to ample practice questions produced by the licensing authority.

AASE appreciates the NCBE attempting to modernize the bar exam to reflect the actual practice of law and decrease the disparate impact on certain populations.  While their goal is virtuous, the current prototypes fall short of satisfying the Testing Task Force’s recommendations.  AASE respectfully encourages all licensing agencies to fully analyze this assessment and consider whether alternative methods of licensure are more appropriate.

Issued: September 6, 2023

Direct inquiries concerning this statement to: Ashley M. London, President, or Steven Foster, Bar Advocacy Committee Chair.

Another Low National MBE Mean

The NCBE released the national MBE mean on Thursday.  The press release is here.  The release highlights the improvement of .2 over last year, which is in a positive direction.  However, I fear too many state bar examiners continue to accept deflated MBE scores without questions.  From 2007-2015, 1 year had below a 142 national MBE mean on a July exam.  Since 2015, 1 year has a 142 or above, and that was 2020 when the NCBE contends the number of takers skewed the statistics.  As someone who taught bar preparation since 2008, I don't believe students now are significantly different than students pre-2015.  Even many of the "get off my lawn" aged professors think students are similar now to pre-2015.  What happened in 2015?  The NCBE added Civil Procedure to the MBE and scores haven't been the same.  With 8 years of deflated scores (with most schools increasing bar prep resources), the NCBE should probably start answering questions.  Bar examiners should step-up and ask questions when individuals' careers are impacted.  Here are a few questions I have (I tried to limit the list):

1.  How did the NCBE take cognitive load into account (ie - adding more material to study) when scaling the MBE in 2015?  (Their stock answer of students studied Civ Pro for essays is unacceptable because not all states tested Fed Civ Pro on essays and level of detail is different for MBE than essays).

2.  When more retakers took the July 2023 bar exam (press release indicates higher percentage), did that artificially decrease the scale for everyone?  

3.  How did the first-time takers compare to previous first-time takers (especially 2018 and 2019) on the anchor questions? 

4.  Is it possible this group of first-time takers performed as well as previous years but the NCBE's lack of accounting for a global pandemic continues to have residual effects on pass rates?

5.  Does the NCBE separate first-time takers and repeater takers performance on anchor questions to create the scale?

My questions may prove unhelpful or even misguided, but the NCBE's lack of transparency raises doubts about scoring.  I could be wrong that students now are just as qualified as previous years.  However, we don't have information to evaluate my questions.  State bar examiners also don't have information or aren't asking these questions.  Alumni lose tens of thousands of dollars in career earnings when not passing the first time.  With that much power over peoples' lives comes even greater responsibility to prove the process works.  We also  shouldn't be complacent waiting for NextGen.  Lets continually ask questions to protect our students and their dreams.

(Steven Foster)

 

Childcare Assistance for Bar Takers: Impactful Opportunities to Assist

When it comes to taking the bar exam, it seems that the primary resource concerns expressed by various stakeholders involve time, money, materials, and dedicated study spaces. However, other impactful factors are important to consider alongside these items, including the impact of working while studying, housing insecurity, food insecurity, and other life quality factors.

One factor that has been on my radar recently is the apparent lack of available resources for childcare assistance for bar takers. Although we often take into consideration the aforementioned factors, the impact of a lack of childcare assistance is one that unfortunately often gets overlooked. However, the time, attention, money, and energy that goes into the care of children, especially young children, can be especially taxing on bar takers who are already strained in terms of resources, therefore it is an important area to address.

In order to assist Academic Support Professionals and other stakeholders in the legal education community, I wanted to share how I was able to assist a bar taker in securing childcare assistance resources in my state (Georgia), and then also wanted to provide you with eight tips you can share with your bar takers to assist them in finding resources to assist them post-graduation while studying for the bar exam.

Childcare Assistance Resources in Georgia: The Georgia CAPS Program

The Georgia Childcare and Parent Services (‘Georgia CAPS’ or ‘CAPS’) program provides financial assistance to eligible families to help with the cost of child care. In assisting bar takers with access to the resources of this program, we had to generally follow the below steps (please note that specific identifying information and other narrative modifications have been made to protect bar takers' identities and for illustrative-ease purposes regarding the general processes and procedures).

1. Determine Eligibility:

Eligibility for the Georgia CAPS program is based on factors such as income, family size, work, or education status. We researched specific eligibility criteria on the Georgia CAPS website and by contacting the Georgia Department of Early Care and Learning (DECAL). The bar taker also had to complete a PreApplication Screener.

2. Apply for Assistance:

Once we were confident in eligibility, it was time to apply for childcare assistance through the Georgia CAPS program. Our bar taker needed to fill out the application which can be obtained online through the Georgia CAPS website or at local CAPS offices. A Georgia Gateway account was required.

3. Submit Required Documents:

The application process typically requires individuals to submit various documents to verify eligibility, such as income documentation, proof of work or education, and information about family size. It is imperative to provide all the necessary documents along with an application.

CRITICAL MOMENT: Our bar taker was no longer enrolled in law school, and this is where the snag in the process typically occurs. However, the eligibility requirements specifically list “State-Approved Activities” which includes online coursework in pursuit of vocational goals, so the critical part of this process was getting a commercial bar prep course approved to satisfy the proof of education requirement. We worked with a commercial bar prep provider to have them provide the documentation to the state regarding the course hour requirements weekly, and the state approved this for educational/vocational purposes. Voila, we were on the way!

4. Attend an Interview (If Required):

Depending on a bar taker’s situation and the local Georgia CAPS office requirements, the bar taker in Georgia may be required to attend an interview to further discuss eligibility and childcare needs. This was not required in our circumstances.

5. Receive Approval and Voucher:

Once our application was approved, we received notice of eligibility along with a childcare assistance voucher. The voucher can be used to access childcare services from approved providers participating in the Georgia CAPS program, so the next step was our bar taker choosing an approved provider.

6. Choose an Approved Provider:

Our bar taker then had to research and select an approved childcare provider who participates in the Georgia CAPS program. The provider must be licensed or exempt from licensure and willing to accept CAPS payments, and our bar taker was able to select one that the bar taker was comfortable with and felt was trustworthy.

7. Provide Ongoing Eligibility Documentation:

After receiving childcare assistance through the Georgia CAPS program, our bar taker was required to provide ongoing documentation to ensure their continued eligibility. This could potentially include updates on the bar taker’s coursework progress, education status, employment, and any income changes.

8. Celebrate Success: It’s always important to celebrate success in helping! Our bar taker was able to complete the process and obtain childcare assistance in the state of Georgia for the full cycle of bar study. This greatly assisted this individual with being able to dedicate study time, and attention, as well as provided the ability to balance financial resources to ensure the taker was in the best position for success on the bar exam.

It is very likely that your state also offers similar opportunities for assistance, and by partnering with your bar vendors to provide documentation and evidentiary support of your bar takers’ programs and time requirements, you will very likely be able to help similarly-situated individuals who need assistance during this critical time.

Eight Tips for Bar Takers Seeking Childcare Assistance

Remember, getting childcare assistance while taking the bar exam will depend on the specific programs and resources offered by the state in which you are taking the exam. Each state will have different options and eligibility criteria. Here are eight tips you can pass on to your bar takers to explore childcare assistance options:

1. Check the State Bar Association Website: Start by visiting the website of the state's bar association. Look for information on any childcare assistance programs they may offer for bar exam takers. Make sure to carefully review bar exam dates and to think carefully about what childcare coverage and assistance you may need.

2. Contact the State Bar Association: If you cannot find specific information on the website, reach out to the state bar association directly via phone or email. Inquire about any available childcare assistance programs or resources for bar exam takers.

3. Inquire with Law Schools and Universities: Law schools in the state where you are taking the bar exam may have programs or services to support their graduates during the exam period. Contact the law schools in the area and ask if they offer any childcare assistance for bar exam takers. Although bar takers often consult their law schools, it is important to utilize all potential resources, so conduct a broad search.

4. Government Assistance Programs: Check with local government agencies or social service organizations to see if there are any childcare subsidies or assistance programs available to individuals in your situation.

5. Scholarships or Grants: Some states or organizations may offer scholarships or grants to help cover childcare expenses for individuals taking the bar exam. Look for any relevant opportunities that you might be eligible for.

6. Law Firm Support: If you are currently employed by a law firm, inquire about any policies or programs they have in place to support employees taking the bar exam, including childcare assistance.

7. Online Communities and Forums: Join online forums or social media groups related to law students or bar exam takers. Other individuals who have gone through similar situations may be able to provide advice and guidance on finding childcare assistance.

8. Plan in Advance: It is essential to start researching childcare assistance options early in your bar exam preparation. Some programs may have limited funding or availability, so planning will increase your chances of securing support. It will also help you in considering budgets in terms of financial resources and your valuable study time.

Remember that childcare assistance programs may vary significantly from state to state and may be subject to budget limitations, vocational and educational requirements, and other limiting factors. It is essential to be proactive in your search for support and explore all available resources in your area. Additionally, be prepared to provide the necessary documentation and meet any eligibility requirements to qualify for childcare assistance programs.

(Scot Goins - Guest Blogger)

Quick Response to MBE Mean

The National Mean MBE score came out on Friday.  You can read the NCBE press release here. My quick research indicates that it is either the lowest or one of the lowest mean scores in history.

I waited a day to think about what I wanted to say about the score.  My thoughts ranged from disdain to despair.  Anxiety skyrocketed for my students who won't get results for a couple weeks.  I wanted to talk about how this can't possibly be the worst set of test takers in history, or the rigidity in licensing continues to ignore at least a year of pandemic education for many of those students.  However, I always fell back on heartbreak.  I am heartbroken and sad for all those who didn't pass.  My disdain for the MBE pales in comparison to graduates' inability to pursue their dreams.  The focus on percentages misses the individual stories of graduates who will lose tens of thousands of dollars in delayed work.  I could write for hours about the problems with the MBE, but until NextGen, it will continue to be an artificial hurdle to practice.  That is heartbreaking.

(Steven Foster)

Colorado Supreme Court Amends Bar Exam Passing Score

This past week the Colorado Supreme Court adopted a new Uniform Bar Exam (UBE) score that is more in line with the majority of jurisdictions. The previous minimum passing score was 276. The new cut score will be 270 starting with the February 2023 UBE. The decision is not retroactive, either for new Colorado UBE takers or for transfers into Colorado.  Here's the link for the details along with a report from the State of Oregon about cut score calculus:

https://www.courts.state.co.us/Media/release.cfm?id=2019

Note: The map below still shows CO as 276 rather than 270 but the table is correct. https://www.ncbex.org/exams/ube/score-portability/minimum-scores/

 

What to Expect When Your Loved-One is Expecting Bar Results!

Here's a terrific handout for bar takers to share with loved-ones as bar exam results are released (created by the Young Lawyer's Division of the Center for Legal Inclusiveness, Chair Marika Rietsema Ball, Esq.):

Impervious to Facts

"Too often facts around me change, but my mind doesn't.  Impervious to new information, I function like a navigation system that has missed a turn but won't re-route,"  writes attorney Mike Kerrigan in a story about "A Sweet Lesson From Pie," WSJ (Sep. 8, 2022).

I suspect that is true of most of us.  But why?  In my own case, my stubborn mind clings to the facts as I know them because, to admit that facts have changed and a new course of "navigation" is required is in someways to admit that I'm a human being, frail in more ways that I wish to admit.

I think that is especially a challenge in legal education and for bar exam authorities.  We cling to the past because that's all we know and, to be frank, sometimes all we want to know.

Take legal education.  We know that learning requires much from our students and from us.  But many of our classes go on despite the new facts that have emerged from the learning sciences. Louis N. Jr. Schulze, Using Science to Build Better Learners: One School's Successful Efforts to Raise Its Bar Passage Rates in an Era of Decline, 68 J. Legal Educ. 230 (2019)., Available at SSRN: https://ssrn.com/abstract=2960192

Take the bar exam.  The best available data suggests that there is a dearth of evidence to support a relationship between bar exam scores and competency to practice law.  Yet we cling to the past. Putting the Bar Exam on Constitutional Notice: Cut Scores, Race & Ethnicity, and the Public Good (August 31, 2022). Forthcoming, Seattle University Law Review, Vol. 45, No. 1, 2022, Available at SSRN: https://ssrn.com/abstract=4205899

I've made lots of wrong turns in my career, my work, and in my life.  To keep on going in the wrong way gets me no closer to where I should be going.  So let's give ourselves and each other the freedom to be changed, the freedom to travel a new path, the freedom to, in short, be curious, creative, and courageous about our work in legal education, on the bar exam, and in life in general.  (Scott Johns).

 

Making Memories that Stick - At Least Thru the Bar Exam

You've heard the quip about "the chicken or the egg, which comes first?"  

Well, as the joke goes, "I've just ordered one of each from Amazon, so I let you know tomorrow!"  

That got me thinking about memorization.  

Most bar takers are really concerned about memorization, particularly because most of their law school exams, unlike bar exams, were open book/open note exams.  But take a look at the word "memorization."  That's a word of action, of a process, of recalling something previously learned.  In other words, at its root core the word "memorization" derives from creating "memories."  So how do you create memories when it comes to learning rules of law?  

Or, to ask it another way, which comes first, memorization or memories?  

Well, I think that the answer to that question is in the question because it's memories that we memorize.  So the key to memorizing is to work through lots of problems, to test yourself with your study tools, to practice retrieval practice, and, in short, to create lots of memories with the rules.  

You see, memorization is just a fancy word for the process of experiencing memories through distributed and mixed practice over time.  So, instead of worrying about memorization as you prepare for your bar exam this summer, focus on making memories (and lots of them).   (Scott Johns)

Dear Practicing Attorneys: Please Stop Giving Our Bar Students Inaccurate Advice.

I still fondly remember the judge for whom I interned as a 3L.  Knowing that bar prep was coming up and sensing my anxiety, he asked me about my plan.  I told him that the bar prep company each day would provide lectures, outlines to read, some more outlines to read, and then finish things off with some outlines to read.  When I told him that the program started just after Memorial Day and ended the day before the exam, he was astonished.  His advice was to save myself all that money, take three weeks off from work, and study from July 4^th until the exam.  He said that would be plenty.

Of all the advice my judge gave me, this was the one bit I did not take.  His guidance was well-intentioned, and I appreciated his attempts to calm me down.  But as the Type-A person that I am, I could not rest without feverishly checking off each scheduled study item.  His was advice I could not take.    

Twenty-something years later, students still receive that advice.  They insist:  “The partner at my firm said that she took just two weeks off for the exam and did just fine.”  The partner professed:  “You’re a smart kid.  You don’t need to do all that work.  Just watch the videos, read the outlines, and you’ll pass.”  Happy to internalize this message so as to mentally corroborate the partner’s flattering assessment, students’ confirmation biases drive them to adopt suboptimal learning behaviors. 

And then they fail the bar exam.

The practicing lawyers who give this advice sometimes believe that the bar exam world is a static place devoid of change.  However, recent substantial reforms severely limit the applicability of their experiences.  Below the fold, I describe those changes and how they require more careful advising. 

Continue reading

Crowd Control - Bar Exam Style

With the move to the Next Generation bar exam, here's an interesting chart, produced using data from the ABA and printed by the State of California, that might caution about placing too much trust in numbers to do the hard work of measuring competency to practice law.

So let's take a closer look. What do you see?  On the horizontal access, we see cut scores for the jurisdictions varying from low too high.  On the vertical access, we see attorney discipline rates corresponding with those jurisdictions. Except for the two outliers, North Dakota and South Carolina, the chart suggests that bar exam cut scores have no apparent relationship in lowering rates attorney discipline claims.  Or, as the State of California put it:

What the scatter plot shows is that attorney discipline – as measured by private and public discipline per thousand attorneys – appears to have no relationship to the cut score. With so many states using 135 for their cut score, the details of the Figure can be somewhat difficult to tease out. The big picture, however, is clear. At a cut score of 135 the rate of attorney discipline ranges from a low of 1.9 per thousand in West Virginia to 7.9 per thousand in Tennessee. Looking across the entire range of cut scores we see strikingly similar rates of attorney discipline in states with cut scores from 130 – Alabama – all the way to 145 – Delaware. California’s rate of discipline (2.6 per thousand) is just over one-half (55 percent) the rate of discipline in Delaware (4.7 per thousand). Given the vast differences in the operation of different states’ attorney discipline systems, these discipline numbers should be read with caution. But based on the data available, it raises doubts as to whether changing the cut score would have any impact on the incidence of attorney misconduct. As with the research conducted by professors Anderson and Muller, this measure of “misconduct” is admittedly limited to cases where misconduct is detected, reported, and sanctioned. There is however currently no better measure of the actual incidence of attorney misconduct or, more importantly, of public protection.[1]

Granted, as California recognizes, attorney misconduct is perhaps a poor proxy for whether bar exam cut scores relate to attorney competency.  But make not mistake.  Bar exam cut scores, by nature of their very arbitrariness, likewise are imperfect proxies for measuring attorney competency.  

Why is this important?  

Well because state supreme courts are now in pitched conversations about whether to adopt the s0-labeled "Next Generation" bar exam.  The chart below suggests caution because it's easy to place confidence in numbers, more confidence that they deserve, like a friend who has betrayed one's trust. As a person trained in mathematics, count me as a skeptic.  Just picking a number out of "thin air," as the range of cut scores suggest, doesn't compute, in my book, as the proper way to judge whether one is competent to practice law.  So as jurisdictions contemplate big changes to a possible next exam format in several years, let's hold them accountability for the math.  Our students and the public at large deserve nothing less. (Scott Johns).

[1] Final Report 2017 Studies, The State Bar of California, available at: https://www.calbar.ca.gov/Portals/0/documents/reports/2017-Final-Bar-Exam-Report.pdf (last accessed Sep. 16, 2021). 

Thoughts on the NextGen Bar Content Outlines

The NCBE released the NextGen Bar's content outlines recently and asked for public comment.  The comments are due Monday April 18th.  They contacted me asking me to spread the word about the notice and comment process and also encouraged me to provide my thoughts on the blog.  I hesitated until now to not cloud others thoughts during the comment process.  I probably should have posted my comments prior to February bar results.  I might (probably not) have been less harsh a few weeks ago.

I will start with my overall view of the NextGen Bar is severely impacted by prior interactions with the NCBE.  I sat in a room with roughly 150 ASPers (probably most of you reading this) at an AASE plenary session where ASPers were supposed to be able to interact with the NCBE regarding massive drops in MBE scores after adding Civ Pro as a topic.  The NCBE representative spoke for about 55 minutes of the hour session.  She took 1 question regarding cognitive load, didn't provide a coherent answer, and the session ended.  No meaningful interaction occurred.  I attended 2 workshops in Madison with the NCBE, which they do graciously provide at no cost to ASPers.  Their hospitality was wonderful, but the substance in the first workshop didn't further schools ability to assist students.  Most of the workshop justified their exams.  The second workshop provided more information.  They did a great demonstration of how they train graders and provided some information about MBE scores.  While the second event still included justifications, it did provide more information for helping students prepare for the bar.  Like many of you, I found the NCBE's claims about "less able students" offensive and lacking any self-reflection.  I felt the NCBE's pandemic response and white-paper justifying the bar exam lacked basic social responsibility to fellow suffering humans.

I could expound on other grievances with cathartic rants, but I should progress to the current topic. I did want to be transparent that my views are based on interactions beyond the current exam restructuring. 

One stated goal from the communication with the NCBE is the bar will test fewer topics less deeply.  I think that is great.  The current MBE tests a depth of knowledge that is significantly beyond minimum competence to practice law.  The goal is great, but I don't trust the NCBE to execute it.  I believe testing skills attorneys use on a daily basis and law that is truly the minimum amount to be competent is an outstanding goal.  I applaud the NCBE for undertaking a task that could radically change the bar exam.  However, goals and ideas alone don't always produce the best results.  Executing a plan to provide a good exam is critical because individual's livelihoods are at stake.  From my experience with how poorly the MBE tests competence, I worry the NextGen Bar will look different but not actually test minimum competence.  My fear is also the NCBE will continue to justify their exam without self-reflection.  A different bar exam isn't inherently good if it continues to test irrelevant skills on a standardized test.

One way to to fail in execution is content.  The content scope outlines illustrate my worry.  I love they are decreasing subjects.  Students across the country will rejoice when Secured Transactions falls off the exam.  Also, no more Family Law or the UCCJEA rules. Decreasing subjects should focus more on the necessary topics for new attorneys.

In theory, substantive cognitive load decreases.  However, I still see 2 problems.  First, the new skills to be tested (Legal Research, Client Counseling, Negotiation, etc.) can't logistically be tested in a real-world environment.  Texas, New York, or Florida can't watch a simulated Negotiation or Client Counseling session for every taker.  Those skills will be tested in some standardized format, which means students will have to learn the "best" answers for those sections.  That still counts towards the cognitive load required to pass the exam.

My second problem relates to the content within the subjects.  The content includes the traditional MBE subjects.  The outline places an asterisk next to areas that must still be memorized.  Glancing at the Contracts outline, nearly everything still must be memorized, including third-party rights, interpretation, and omitted terms.  Business Associations also seems to need memorization of sections (ie - LLCs) that should be state specific statutory law.  The amount of substantive memorization may decrease due to less subjects, but some subjects still seem to require memorization.  I believe some of the memorization is still beyond what regular "competent" attorneys know.

My problems aside, I do love that common law crimes are no longer tested.  Virtually none of Crim law must be memorized.  Significant portions of Real Property doesn't need to be memorized, especially future interests.  I would throw future interests out completely, but no memorization is a compromise.  Civ Pro requires memorization but most of Evidence doesn't.  There does seem to be effort to decrease content, but I think more could be taken out.

If I merely read the NextGen Bar's content scope outline with their goals, I think it could be a reasonable and relevant exam.  However, I am skeptical of the NCBE based on prior interactions.  I question whether the execution will follow the goals and whether this becomes another standardized mechanism to exclude diverse populations.  I hope I am wrong.

(Steven Foster)

Notice and Comment on NextGen Bar Content

The NextGen Bar Exam's preliminary Content Scope Outlines are out.  The announcement is below.  The NCBE is also asking for comments on the outlines.  You can use the links in the announcement to provide comments.  I encourage everyone to take advantage of this opportunity to advocate for our students.

 

"NCBE is pleased to announce the publication of the preliminary Content Scope Outlines for the next generation of the bar exam. NCBE requests public comment on the outlines from our colleagues in the legal community by Monday, April 18. The new bar exam is expected to launch in approximately five years. Please feel free to share this announcement and request for comment with your colleagues in practice, the judiciary, and legal education.
 
The Content Scope Outlines delineate the topics and lawyering tasks to be assessed within the eight Foundational Concepts and Principles (subjects) and seven Foundational Skills established through the work of NCBE’s Content Scope Committee, a group of 21 dedicated legal professionals, including legal educators, deans, practitioners, and bar examiners. The focus of the topics and skills to be tested are those that are most important to the practice of newly licensed attorneys.
 
These subjects and skills are based on the input of nearly 15,000 members of the legal community who responded to NCBE’s 2019 nationwide practice analysis survey. The subjects to be tested are Civil Procedure, Contract Law, Evidence, Torts, Business Associations, Constitutional Law, Criminal Law, and Real Property. The skills to be tested include legal research, legal writing, issue spotting and analysis, investigation and evaluation, client counseling and advising, negotiation and dispute resolution, and client relationship and management.
 
Thank you for your interest in the future of the bar exam. To submit comments by April 18, please visit the next gen website."

Academic and Bar Support Scholarship Spotlight

Recent SSRN posts on academic and bar support:

1. Catherine Martin Christopher,  Modern Diploma Privilege: A Path Rather Than a Gate (SSRN Post, October 5, 2021). 

From the abstract:

This article proposes a modern diploma privilege, a licensure framework that allows state licensure authorities to identify what competencies are expected of first-year attorneys, then partner with law schools to assess those competencies. Freed from the format and timing of a bar exam, schools can assess a broader range of competencies over longer time horizons. This will allow the development of law school curricula aimed at preparing students to assist clients rather than to pass the bar exam. The modern diploma privilege is structured as an ongoing partnership between licensure authorities and schools, which means that changes can be easily made to the list of desired competencies and/or the assessment methods. This in turn allows for a more nimble licensure mechanism that can quickly adapt to changes in the evolving market for legal services.

2.  Katharine Traylor Schaffzin, First-Generation Students in Law School: A Proven Success Model, 70 Arkansas L. Rev. 913 (2018).  

From the abstract:

This article addresses the ever-increasing population of first-generation college students and the academic challenges they face both in undergraduate school and in seeking to matriculate to law school. This demographic has been heavily studied at the undergraduate level, but very little data is available about the challenges and success of first-generation college students in law school. The article describes the best practices for the academic success of first-generation college students as researched and implemented by various colleges and universities. It also summarizes the findings of the only study done on the experiences of first-generation college graduates who matriculated to law school.

This research serves as the backdrop for the description of a unique program with proven success directed toward securing the academic achievement of first-generation college students in law school. The University of Memphis School of Law Tennessee Institute for Pre-Law (TIP) program is decades old and has been recording the successful outcomes of such students. This article analyzes data collected since 2012 on the academic outcomes of first-generation college graduates who participated in TIP to conclude that the program leads to successful results for these students in graduating law school and passing the bar exam. The article details the program itself and explains how a law school can implement the promising methods uncovered at the undergraduate level. It offers TIP to readers as a proven intervention and success model for law schools seeking to ensure the academic success of first-generation college graduates in law school.

Recent book:

Charles Calleros (New Mexico),  Law School Exams 3rd edition, VitalSource (2021).

From the publisher:  

Law School and Exams: Preparing and Writing to Win, Third Edition is the third edition of a popular book whose first edition Bryan Garner reviewed and judged to be “the best on the market.” It combines:

1. 1. Clear and comprehensive explanations of study and exam techniques
   2. Numerous illustrative samples that are truly instructive
   3. Twenty in-class exercises or take-home assignments on everything from case briefs to essay and multiple-choice exam questions.

Comprehensive and self-contained, the Third Edition is suitable for use as the textbook for a sophisticated Prelaw course, 1L Orientation, or a 1L Academic Success course. Alternatively, incoming freshmen can work through it independently over the summer to be optimally prepared for law school in the fall.

(Louis Schulze, FIU Law)

A Dean Speaks Out

I've been meeting with unsuccessful bar takers, and I'm finding that it is increasingly difficult to explain holistic relative-rank scoring, in which what appears to meet competency standards is judged incompetent.  

I realize that the NCBE and jurisdictions say "trust us" because we use statistical equating and scoring methods to standardize written scores based on MBE distribution data including median and mean MBE scores and standard deviations.  

But, frankly, it seems unfair to toss some written exam answers, especially legal writing performance test answers, into the 1 or 2 out of 6 "buckets" when the pool of applicants have already undoubtedly proven their merit through earning doctoral juris degrees.  

So, fancy this, a dean speaks out, suggesting that the bar exam as a rite of passage is not moored to its stated goal of measuring entering level attorney competency but rather tied to 1920-era exclusionary politics.  

I'll let the dean speak for himself:

"When we started seeing diversity increase or people from underrepresented communities — mostly people of color and recent immigrants, trying to become lawyers — then all of a sudden the ABA (The American Bar Association) and other bar organizations were doing whatever they can to keep them from being lawyers," Niedwiecki explained. "The written bar exam became a requirement of the ABA at that time. So that's when we started seeing all these written bar exams. Before that there were oral exams... apprenticeships, there were other ways to become licensed. I think we have to go back to those days knowing that the bar exam really kind of was back in the '20s rooted in exclusion." Niedwiecki, A, "Why a Mitchell Hamlin's Dean is Calling for an End to the Bar Exam," KARE-11 TV (Sep. 30, 2021).  (S. Johns).