Law School Academic Support Blog

Editor: Goldie Pritchard
Michigan State University

Monday, October 14, 2024

What Are We Made For?

I know you all saw the Barbie movie, but if you haven’t, please go do that now. I’ll wait. For those of you who did not follow my directions,[1] there are spoilers ahead.

The Barbie movie is about a lot of things, some aspirational (like a completely matriarchal society), and others just simple human quests to understand the point of being alive.[2] Generally speaking, our protagonist is a standard Barbie who suddenly becomes aware that she is living in a fictional world and isn’t quite sure why she is there or her purpose. She goes on a journey to try to find answers and (your interpretation may vary) figures out that the journey to seek answers is the key to being human and while being human has its downfalls (emotions, aging, pain, the patriarchy, etc.), it is what she needs to be.

Those of us in ASP are similarly on a journey to figure out our purpose and place in law school academia.  The status of ASP folks is a little all over the place. Some of us are Deans; some are faculty; some are staff; some are at-will employees. But extremely few of us are tenured or even are afforded tenure-like job security. I have written about this many times. But there was a brief glimmer of hope over this past summer when the ABA proposed the following amendments to Standard 405 (as well as Definition 9 which defines "full time faculty" under 404 and 405):

“(c) To attract and retain a competent faculty, and to secure academic freedom as outlined in Standard 208, a law school shall:

(1) Afford all full-time faculty members, other than visiting faculty members or fellows on short-term contracts, tenure or a form of security of position reasonably similar to tenure.

    (i) “A form of security of position reasonably similar to tenure” means such security of position as is sufficient to attract and retain a competent faculty and to ensure academic freedom. Providing such security of position requires that, following an appropriate probationary period, a full-time faculty member may be terminated or suffer an adverse material modification to their contract only for good cause. Such good cause determination shall be made only after the full-time faculty member has been afforded due process.

    (ii) Full-time faculty members need not all be subject to the same rules regarding tenure and security of position.[3]”

Cue up Etta James’  “At Last”[4]

The ABA goes on to say that a change in ASP status is needed because (among other reasons), "When faculty members lack security of position, it undermines the quality of legal education and harms law students. Non-tenure track faculty members in the areas of academic support/success and bar readiness report being afraid to help students who have substantive questions about doctrinal areas of law because they fear that a tenured faculty member will complain to the dean that the academic support faculty member is trying to teach Torts, Contracts, or some other substantive doctrinal area of law. This fear has led many faculty members without security of position to limit their assistance of students to the “skills” aspect of teaching which frustrates the efforts of students who seek out academic support to ensure their own academic success through full understanding of the substance of legal doctrine as well as the skill of applying legal doctrine."[5]

I would also add that people who have an entire faculty (or just one Dean) determine their employment also lack a voice and cannot freely engage in political speech or other aspects of academic freedom enjoyed by those who need not fear offending anyone in order to keep their job. This proposal might be the end of that boxed in feeling.

But let's wait a moment before rejoicing.

In ASP, we are not all considered faculty (although by other ABA current and proposed metrics, we should be based on our roles and responsibilities). For ASP professionals who are still considered staff by their institutions, the ABA then goes on to take away any cause for celebration by including an explanation of the parameters of the proposed change to Standard 405 that says,

“A significant addition to Standard 405(c) is the requirement that the director or supervisor of the academic success, bar preparation, field placement, and legal writing programs have tenure or a form of security of position reasonably similar to tenure. This requirement does not necessitate transforming a staff position into a faculty position: the requirement can be satisfied by having a faculty member with tenure or security of position reasonably similar to tenure overseeing these programs. For example, if a law school’s academic success/support unit is comprised of staff members and these staff members are supervised and overseen by an associate dean – or other faculty member who has tenure or security of position reasonably similar to tenure – the law school can satisfy Standard 405(c).”[6]

In other words, under this proposed “change”  law schools don’t have to convert ASP staff to faculty members to follow this standard. It is sufficient if you appoint an anointed adult to supervise them and that this person alone has tenure. This seems like relegating ASP to be the help rather than family.

Again.

And here’s the thing: we should be family. We are the folks that bolster rankings by improving bar pass rates (Standard 316), and lowering attrition, and closing gaps before the bar is ever undertaken (Standard 308). Law schools cannot meet other ABA Standards without us. We often are the faculty who teach Professional Identity Formation (Standard 303), and we are absolutely required under Standard 309 in order for a school to be ABA Accredited,

“Standard 309. ACADEMIC ADVISING AND SUPPORT

(a) A law school shall provide academic advising for students that communicates effectively the school’s academic standards and graduation requirements, and that provides guidance on course selection.

(b) A law school shall provide academic support designed to afford students a reasonable opportunity to complete the program of legal education, graduate, and become members of the legal profession.”[7]

So, if we are not fundamental to the essence of a law school, what are we made for?

Billie Eilish (and her talented brother) wrote the multiple award winning “What was I made for?” song for the Barbie movie, and in it, they ask,

“…

Takin' a drive, I was an ideal

Looked so alive, turns out I'm not real

Just something you paid for

What was I made for?

Something I'm not, but something I can be

Something I wait for

Something I'm made for

Something I'm made for.[8]”

Please feel free to send your comment (right now because comment closes today) to the ABA.[9] Add your voice to the AASE/AALS comment filed earlier.

Tell the ABA to stop relegating ASP to second class status.

We do more.

We impact more.

We were made for more.

(Liz Stillman)

[1] Feel free to get on the non-follower line behind my kids, husband, and pets….

[2] I agree that was a fast switch, but again, I am not asking for almost 2 hours of your attention here.

[3]https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/council_reports_and_resolutions/aug24/24-aug-notice-comment-standard-405.pdf

[4] https://www.youtube.com/watch?v=1qJU8G7gR_g

[5] https://www.americanbar.org/content/dam/aba/administrative/legal_education_and_admissions_to_the_bar/council_reports_and_resolutions/aug24/24-aug-notice-comment-standard-405.pdf

[6] Id.

[7] Id. 

[8] https://en.wikipedia.org/wiki/What_Was_I_Made_For%3F

[9] Per the ABA, "We solicit and encourage written comments on all the proposals listed above. Please note the changes to the submission instructions as follows: All written comments should be addressed to David A. Brennen, Council Chair, and sent electronically as a .pdf attachment to [email protected] by October 14, 2024."

 

October 14, 2024 in Current Affairs, Encouragement & Inspiration, Film, News, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, June 17, 2024

The Importance of Being AASE

I love Oscar Wilde’s The Importance of Being Earnest. I mean who doesn’t love a fast talking piece that is entirely a play on words from beginning to end? It was an oasis in an otherwise uninspired year of high school English reading for me.[1] As one would guess, it is convoluted and hard to wrap your brain around. It all seems a bit unknowable-almost the same way the breadth and depth of academic support work can get lost in translation. We all know how exhausting it is to tell people what we do over and over.

I also think most people I have met in law school academia find it “interesting” that there is a national organization of academic support educators.[2] But as we move towards more status equity, having this resource will prove imperative. We need to find a way to get to a critical mass of tenured and tenure track positions (like Legal Writing faculty have done) in order to gain better equity footing across the board. This is where having a national organization is key. As a national organization, we can collect and share voluminous data about how ASP faculty (and non-faculty!) are treated, paid, supported, unsupported, and fit into the law school community. In short, we can, if we work together, make long overdue progress. Here are some things we can do to move the needle forward:

  1. Amplify each other. Tell another AASE member’s Deans or other significant administrators about our colleagues. Did you like a presentation, article, book, helpful tip from them? Great. Send an email. I know a lot of people do this, but ASP folks, let it rain!
  1. Use the AASE website as a place where we:
    1. Share resources;
    2. Post jobs;
    3. Post scholarship;
    4. Post teaching resources;
    5. And (copying entirely from the Legal Writing folks) accumulate resources to further this advocacy.[3]
  1. We will be doing the AASE Survey again this year. Fill. It. Out. Data is everything in this fight. If we have better participation, we can use the data for salary and status benchmarking in a much more credible way. And then we need to disseminate the data widely-so that even the fancy schmancy consultants who do salary benchmarking can find it without much effort.[4]
  1. Consider advocating for a place at the rankings table. I’ve written about this before and I stand by it: if we are an asset to our school then they are going to want to put a ring on it.[5]
  1. Finally, consider advocating for a place at all the tables that have not had a seat for us before. We know what we do and how well we do it, but we need to shout about that from the rooftops. We need to showcase our brand. It is not something that comes easily for us, but we really need to be a bit more braggy about our accomplishments-because they are abundant, and people ought to know.

One great quote from The Importance of Being Earnest is, “[e]ven before I met you I was far from indifferent to you.” Let’s make this the new narrative about Academic Support.

(Liz Stillman)

 

[1] That kind of word choked, fast moving dialogue was something I loved about Gilmore Girls as well. 

[2] Although there are many specialized groups in law school academia that seem even more narrowly focused.

[3] Comme ça (as the French would say): https://www.lwionline.org/resources/status-related-advocacy

[4] Yes, I’m still a little salty about not having any salary benchmark when it was reviewed last year at my school despite having the AASE survey results in the freaking building.

[5] Apologies to Beyoncé.

June 17, 2024 in Current Affairs, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, September 25, 2023

Invisibility

Many of you probably received an email from AALS last week with a link (and “unique PIN”) to a “Faculty Survey.”  The email said,

The Association of American Law Schools is interested in your experiences as a law school faculty member. AALS wants to know more about you, your career trajectory, current workload, time allocation across your various responsibilities, and perceptions of tenure. We are asking you to take part in the American Law School Faculty Study…

The survey itself, being conducted by an outside vendor, (NORC) has the following preamble (again, the bold is in the original):

“This survey focuses on the experiences of individuals who currently serve in the position of law school tenured, tenure-track, long-term contract, clinical, or legal writing faculty.”

It is a well-established canon of construction that, “the expression of one thing implies the exclusion of others (expressio unius est exclusio alterius).”[1] So, the preamble alone should have made it clear to me that ASP and Bar Prep faculty members were not their intended audience-and yet, it was sent to all of us. If I had not checked off “long term contract,” my survey would have ended right there. Luckily, a colleague alerted me to this before I started, and I was able to voice my displeasure at being intentionally excluded as part of my response. Otherwise, I would have remained invisible.

As we know from the AASE Survey last year, not all of us could click on long term contract and avoid being entirely canceled from being considered faculty by an organization that our institutions are likely members of and actually has an Academic Support Section[2]. In fact, only 26% of AASE respondents are on multiyear contracts and 17% have presumptively renewable contracts. 47% of respondents are at-will employees and another 11% have year to year contracts.[3] This means that less than half of our ASP colleagues would be eligible to participate in this survey. Surely, our experiences are as relevant as other traditionally non-tenured faculty such as clinical and legal writing. While there has been progress in tenure for these other groups, ASP tenure (or tenure track) is currently unavailable to 92% of professionals who responded to our survey.[4]

My esteemed colleague, Matt Carluzzo, who is Assistant Dean of Students and Academic Success at Villanova University Charles Widger School of Law responded to NORC with an email where he expressed his disappointment and went on to say, “[M]any law schools still see and accordingly treat ASP as an afterthought - something necessary, but still very ‘other’ … I was initially disappointed (though not surprised) when there was no "academic success/support" option listed on the opening page.  I was genuinely shocked, however, when upon selecting "Other," I was instantly directed to the curt, "Thank you for your time today" completion screen.  Apparently this survey is not for ASP professionals.  This is hard to interpret as anything other than yet another example of ASP being either unintentionally overlooked, or intentionally excluded…Your website says that AALS ‘hired NORC to learn more about law school faculty hiring, voting rights, tenure policies, and other key issues[5].’  In my opinion, this is a key issue that is blatantly overlooked and/or ignored.  Any doubt, disbelief, or resistance to this idea is contradicted by the old cliche: the proof is in the pudding.” I could not have said it better. We await a response from AALS, NORC, and perhaps even the AccessLex Institute (who was another sponsor of the survey).

In the meantime, I am convinced that when clicking “other” brings you to a dead end, it is not a good look for an organization that claims that their “...mission is to uphold and advance excellence in legal education. In support of this mission, AALS promotes the core values of excellence in teaching and scholarship, academic freedom, and diversity, including diversity of backgrounds and viewpoints, while seeking to improve the legal profession, to foster justice, and to serve our many communities–local, national, and international.[6]” I would also add that the introduction to the survey expresses AALS’s interest “in examining the work-life balance and career trajectories of law faculty.”[7] 

If the opinions of legal writing and clinical faculty merit consideration, ASP faculty opinions should not be overlooked and disregarded. While the doctrinal faculty that seem to be the target of this survey do not always know all that we do in ASP, they no doubt are glad it is done.  Their students certainly are. We should be seen and heard. We deserve-—no, wait—we have earned better.

If AALS truly wants to know more about the “career trajectories of law faculty,” why not study the folks who have nowhere to go but up?

(Liz Stillman)

 

[1] https://judicature.duke.edu/articles/a-dozen-canons-of-statutory-and-constitutional-text-construction/

[2] However, there were some issues about ASP’s inclusion at the AALS conference this past January as well, see, https://lawprofessors.typepad.com/academic_support/2023/01/academic-support-programs-should-be-included-in-us-news-rankingsmaybe.html

[3] Please feel free to contact any of us who serve on the AASE Assessment Committee for the full survey report: https://associationofacademicsupporteducators.org/committees/assessment/

[4] See, note 3.

[5] https://www.norc.org/research/projects/2023-american-law-school-faculty.html

[6] https://www.aals.org/about/mission/

[7] https://www.norc.org/research/projects/2023-american-law-school-faculty.html

September 25, 2023 in Meetings, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, May 22, 2023

Best Practitioners

Greetings from Santa Clara, California, and 10th Annual AASE conference! The sun is shining, and it is amazing to see everyone-the people I have missed in our pandemic years as well as people I had not met in person before today (like the amazing editor of this blog, Steven Foster!)

Here are the things I've learned so far (today was the day for "newbies" to learn the ins and outs of Academic Support):

  1. There are palm trees here-but they are not indigenous to this area. But they are so pretty swaying in the wind. I know they'd not survive a New England winter, but I wouldn't mind giving a try....
  2. ASP People are the best people-actually, I already knew that, but proof of this fact was undeniable today. We are the kindest, most generous, and collegial academics out there. And if you argue with me about that, I'll most likely ask you for your sources and then have you frame a counterargument because that is what we do, but I won't be thrilled about it.
  3. Although I am far from a newbie, I was bolstered by listening to the most respected folks I know tell me what their process is, and even more exciting: it is my process too!!! Which is not to say I didn't learn amazing new things, but I am so happy I am engaging in best practices. Phew!
  4. We are doing world class scholarship and lifting each other up with it. This is wonderful!!
  5. I cannot wait to see what else (and who else!) I will encounter tomorrow.

I am looking forward to spending more time learning from, as well as hanging and laughing with the amazing community. We value each other when we aren't universally valued in other realms. we are family.

(Liz Stillman)

 

May 22, 2023 in About This Blog, Meetings, Professionalism, Program Evaluation, Teaching Tips | Permalink | Comments (0)

Thursday, April 13, 2023

Survey reminder no. 3,564,722

Last week, we sent out another email with the individual and institutional survey links to all AASE members. If you didn't receive it, please email me at: [email protected] and I'll get it to you!

The data that we amass as a result of this survey will help our profession know a number of things:

  1. Who we are: who are the ASP professionals in our nation's law schools
  2. What we do: so, so much, but more specifically we will have information on what classes we teach, workshops we offer, bar prep (during and after law school), orientation programs...really everything we offer to our students.
  3. How we are valued, classified, and compensated. This cannot change if we do not know the baseline.
  4. How we spend our time in these roles, doing all this work.

 As promised (threatened?), here is a limerick for the occasion:

There once was a survey from AASE     

That didn’t take up all that much space

It asked for the info we need

To help us succeed

In making our tenure track case!

The deadline to answer (APRIL 14TH!!!) is TOMORROW!!!.

Please do not make me resort to sonnets.

(Liz Stillman)

April 13, 2023 in Meetings, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, April 3, 2023

Have I Mentioned the Survey(s)?

You and I both know that I have mentioned it (a number of times). Last week, we sent out an email with the individual and institutional survey links to all AASE members. If you didn't receive it, please email me at: [email protected] and I'll get it to you!

The data that we amass as a result of this survey will help our profession know a number of things:

  1. Who we are: who are the ASP professionals in our nation's law schools
  2. What we do: so, so much, but more specifically we will have information on what classes we teach, workshops we offer, bar prep (during and after law school), orientation programs...really everything we offer to our students.
  3. How we are valued, classified, and compensated. This cannot change if we do not know the baseline.
  4. How we spend our time in these roles, doing all this work.

 I have even composed a Haiku to inspire you to respond (I think we forgot to add poetry as a category of ASP work on the surveys, but nonetheless):

Please take the survey,

the data will help us all,

as professionals.

The deadline to answer (APRIL 14TH!!!)is coming sooner than you think. Please do not make me resort to limericks.

(Liz Stillman)

 

April 3, 2023 in Miscellany, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, March 27, 2023

BOLO

Please be on the lookout for the 2023 AASE surveys later this week. We plan to launch the individual and institutional surveys on Wednesday and have them remain open until April 14th.  We are looking to collect data on who we are, who we serve, our status in the academic hierarchy, and what we do both inside and outside the ASP/Bar Prep paradigms. We will be presenting our findings at the AASE 10th Anniversary Conference in May.

Please, everyone, fill out your individual survey when you receive it!  It is entirely anonymous. If you are a program director, you get to fill out two surveys (yay!): one for yourself and one for your school.

Our quest for equity begins with the collection of data. We are valuable members of every law school’s faculty team, and while it seems unsavory (and sometimes outright unfair) to have to prove ourselves to get the respect (and salary) we deserve, we must. More participation gives our data more credibility for use later on.

Be counted and seen!

(Liz Stillman)

March 27, 2023 in Meetings, Program Evaluation | Permalink | Comments (0)

Monday, March 13, 2023

Numbers Game

Academic Support and Bar Prep educators are among the hardest working people I know. We are selfless student supporters. We are scholars. We are generous with our work, praise, and time. As a group, we would probably be voted “Most Likely to go Above and Beyond” in a fictional law school yearbook. However, one accolade we are not going to get in this fictional yearbook (at least at this moment) is “Most Likely to get Tenure.”

We need to go above and beyond on our own behalf to gain the job equity, security, and salary that recognizes the work we do.  We need to take a small fraction of our focus and use it for ourselves and each other.

In about two weeks, you will get two surveys from AASE. One is for you individually, and the other for your institution. If you are the director of your program, you should fill out one of each, if not, please only fill out the individual survey and nag your director to fill out the institutional survey for your school. If you don’t see the survey by April 1st, please contact AASE at: [email protected] and we will send you the surveys.

Here’s the thing, we all need this data. We need to know who we are and how we are doing as a group. We need to know what job security looks like for us --or if there is any at all. We need to know how much we are being underpaid compared to other groups of law school faculty. Knowing what we all do both in and outside of the ASP realm is important. Knowing what we teach, how often, and when we teach it, is incredibly valuable information. I know it seems intrusive, and my mother would often say that asking about salary is just “tacky,” but our institutions will be looking for this information when we propose a change.

Data is how the legal writing community successfully waged their tenure battles. Numbers seem like unlikely armaments, but at the moment, they are the tools we need. When the results of the survey are presented at the AASE conference in May, please do not be the person listening and thinking, “they haven’t captured my situation.” We want to capture you (not in a kidnapping or any other creepy way, you know what I mean….hopefully…).  We want the team photo of "ASP educators with tenure" to be big enough to need a full page spread in future yearbooks.

Getting the appropriate and earned equity, security, and pay for our community will be a numbers game. Please play.

(Liz Stillman)

March 13, 2023 in Encouragement & Inspiration, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, February 13, 2023

37 Pages of Love

I have spent the last few months helping to draft an internal document for my law school that is supposed to evaluate the current state of the entire upper-level curriculum and make some recommendations based on those assessments. I will preface my list below by stating that my school has been amazingly cognizant of the issues we’ve raised, but my little committee also did some outside research that identified these general issues. Writing this report has been both an overwhelming and incredibly nebulous task, but here are some things I’ve learned on the way to dropping off those 37 pages of love off to the higher powers:

  1. Some of our recommendations aren’t going to matter much if the NextGen bar exam is adopted by our state bar. No one will need to take Secured Transactions anymore….
  2. Students in academic distress will tend to stay there (Newton’s Law of Academic Warning?) because while we are (understandably) concerned about them passing the bar, we are sending them to classes that are quite similar to the ones that caused the initial distress. More big classes where the curve is required are not the answer to doing poorly in big classes where the curve is required. It is like giving students who are stuck in a ditch a shovel rather than a ladder.
  3. We should try to ensure that every student, and especially those in academic distress, has as many different types of legal instruction as possible: doctrinal, skills-based, experiential, transactional, etc. Students who are limited will not see themselves as lawyers, just mediocre law students. This isn’t good for their confidence while still in law school and it could honestly exacerbate mental health issues. If a large class, with a curved exam, that employs lectures doesn’t work for a student, why make that a big chunk of what they need to take to continue in law school?
  4. Smaller classes would most likely benefit both students and faculty. I think this is particularly true of classes required for students in academic difficulty, but I do not want my report to be the reason our Dean is sneaking out of the building to buy lottery tickets. Sure, more funding for all of this would be great, but then law school tuition would be out of reach for most and that is exactly what we are trying to avoid.
  5. While writing this report was time-consuming and sometimes frustrating, it is worthwhile to take the time to see where we are and make recommendations (big and small) that can take us to a better place. Sure, some of what we recommended was purely aspirational, but if the Dean gets the Powerball jackpot, you never know what is possible….

(Liz Stillman)

February 13, 2023 in Bar Exam Preparation, Encouragement & Inspiration, Learning Styles, Program Evaluation | Permalink | Comments (0)

Monday, January 9, 2023

Academic Support Programs Should be Included in U.S. News Rankings...maybe

Happy New Year ASP Blog Readers! We are back!

Earlier today I was in a meeting with colleagues who told me that several publishers had not brought ASP/Bar Prep publications to the AALS meetings because, (and keep in mind that this probably hearsay squared), “AALS is for doctrinal faculty.” AALS abolished the distinction years ago, but perhaps that message has not reached all sectors. And while I could easily lament another occasion where academic support is overlooked and excluded, today I have another proposition.

I have written in the past about how U.S. News Rankings count the work of academic support and bar prep professionals (bar pass rate!!!), but they do not evaluate the programs themselves. This is, I have argued, essentially taxation without representation.

Recently a few schools that probably do not sweat the bar pass rate (let’s be honest here, it is always going to be in the over 90% area for them), have decided not to engage in rankings. These schools just don’t need the credential to boost their marketability or community standing. They already have all the name and prestige recognition they need. They just shuffle among the top tier like a tableau of rich invitees at a Gatsby event. But, as I tell students fairly often, 90% of the class in not in the top 10%. So too for law schools -- as a vast majority of schools are not invited to the West Egg shindigs.

After attending an amazing conference organized by the New England Consortium of Academic Support Professionals where we discussed job security, equity, and even reached for the brass ring of tenure, I am convinced that having academic support programs ranked by U.S. News might be a step in the right direction.

Here are my top three reasons:

  1. This would be another metric for schools looking to gain status, meaning that schools that really do need a boost can get one, and
  2. It might shift a power dynamic to a successful (and therefore ranked) academic support program’s professionals to seek better job security (contracts where they are at-will employees, presumptively renewable contracts for those on a year-to-year contract, and tenure down the road.) A school that gains prestige because of a ranked ASP program would want to protect that asset.
  3. ASP professionals work extremely hard-we teach more, we meet more, we write as much (if not more), and we are often asked to take on responsibilities that are similar to doctrinal, legal writing, and clinical faculty. We deserve the recognition-beyond the amazing way we honor each other in our community.

But there are some downsides:

  1. More scrutiny doesn’t always reveal only good things. We might put folks with very little job security in a more precarious position and introduce metrics that are not necessarily indicative of quality academic support. This might turn out to be another area where BIPOC professionals are not fairly evaluated.
  2. ASP will now be tethered to raising or maintaining a ranking--which is not the point of ASP. This might distract us from our students, who are the reason we do what we do.
  3. Being tied to the bar pass rate more directly may not be fair since some of the variables that control bar pass rate are not within the control of ASP. We cannot overcome a poorly admitted class, or a pandemic, for example.

I invite debate on this idea. I would also happily invite the beginning of a national movement of ASP professionals to work together toward more equity and job security. If we take any page from legal writing, the one I believe is foundational would be that we gather our data and work together.

(Liz Stillman)

January 9, 2023 in Miscellany, Professionalism, Program Evaluation | Permalink | Comments (0)

Monday, June 13, 2022

But do you like me, do you really like me?

My Law School course evaluations arrived without warning or fanfare in my inbox Saturday afternoon. The subject line, “spring 2022 course evaluations” popped up on my phone while I was sitting at the optometrist’s office picking out a new pair of glasses that would (ironically) make reading things on my phone easier. I had received my course evals for my undergraduate course a few weeks back and they had come, pre-read by the department chair, with her encouraging words that slowed my heartbeat a bit before diving in. But the law school ones just showed up as an attachment: unannounced, and to be honest, panic inducing. I wasn’t ready. We tell students when the grades will be released, so perhaps a similar warning may be warranted. As it was, I held my breath and clicked.

To be fair, I had thought the semester had gone well (there are always a few students who are unreadable, but they didn’t seem hostile), so I should not have started to sweat when this email appeared. But I was grateful for the air conditioning at the eyeglass shop, nonetheless.  Although the literature is a bit all over the place, there seems to be a grudging consensus that, “… student evaluations as currently constructed are strewn with gender and racial biases. Instructor attire and weight has impacts on student evaluations, too. In short, there is a lot of noise in student evaluations that have nothing to do with teaching and everything to do with student biases.”[1]  I also think that the anonymous on-line iteration of course evaluations has made students a little more, um, blunt.

I have had evaluations that commented negatively on my snacking (I was pregnant, and it seemed better to eat my baggie of Cheerios rather than puke on students), my sense of humor, and my clothing choices (which honestly felt more like body shaming). It all feels a little middle school-ish to me because this is the documentation of what people might be saying behind your back. I also remember my favorite comment of all time, “Condragulations Professor Stillman, you are a winner.” Using a RuPaul’s Drag race reference made me feel really seen and I treasured it.

Are some evaluations biased or just plain mean? Probably. But discounting them entirely also negates the good ones (luckily far outnumbering the bad, I’m sure). I also need to read them to know if I am connecting with students. I want to be sure that I am respectful of opposing viewpoints (not my strong suit, really). If I don’t care what the students think (about some fundamental things, not my wardrobe per se), then I am not teaching for the right reasons. If the evaluations can legitimately assess my teaching, then this is information I need. If not, they give students power over non-tenured faculty that they do not deserve.

Evaluations are truly a double-edged sword. Make no mistake though, they may still be a weapon.

(Liz Stillman)

 

[1] https://abovethelaw.com/2022/06/making-student-evaluations-more-meaningful/

June 13, 2022 in Professionalism, Program Evaluation, Stress & Anxiety | Permalink | Comments (0)

Monday, April 25, 2022

Lost Opportunities

This semester I changed up my assessments for my undergraduate law class. In the past, I had done oral arguments as a final assessment, but after witnessing paralyzing anxiety from more than a couple of students last semester, I decided that I was assessing mental health rather than legal argumentation skills. No one should be graded that way. So, this semester, students are writing a judicial opinion (pretending to be a U.S. Supreme Court justice) in the case of Carson v. Makin.[1] This is a fun case for my undergraduates because it took place here in the First Circuit (nearby in Maine) and it is about high school (also temporally nearby for undergrads). The case is about Maine’s program for students in very rural areas that do not have a local public high school. Maine’s law allows parents to choose another public school in a different district, or a private school so long as the private school meets certain criteria in terms of state required curriculum, attendance, and our sticking point: that the school is “non-sectarian”.[2] The case is a great example of free exercise clause litigation and students are really getting into it, but the very complicated issue of standing is one I have had to take off their plate because it is a bit too much for students who have not taken a course in Federal Courts. Essentially, the plaintiffs are parents who would like to send their kids to sectarian schools but because of the Maine law, they haven’t even tried to use the tuition assistance program[3]. The schools that the parents want to send their children to have not agreed to follow Maine's other requirements either[4]. So, you may be asking, how have they been injured? The attorneys for Maine asked this as well, in more than one case, and each time the District Court and First Circuit found that there was standing because, “[T]he plaintiffs’ injury in fact inheres in their having lost the “opportunity.”"[5] It seems a little like tap dancing in the rain to find standing here, but there it is: a lost opportunity is sufficient injury to get the case before a court.

This decision made me think that we may be injuring our students who are on Academic Warning, Probation, guided curriculum, or whatever your school might call it. We do, of course, intend to help these students in terms of bar readiness and supervision to prevent further academic mishaps. We have a compelling academic interest in having students take this path. Our studies and experience show that it works. I really have no doubt that our process does improve our students’ chances overall. To that end, we have students take bar tested courses like Evidence, Commercial Law, Family Law, and Trusts and Estates once they have a GPA below a certain threshold.

But…these students are required to take another set of large, grade-curved classes that tend to have one high-stakes summative assessment at the end. This might be where things initially went wrong for them, so more of it may just dig the hole deeper for some. We also occupy their schedules with required courses that monopolize their time and credits each semester. Students in academic difficulty do not often get the green light to take a credit overload. There is less space, after satisfying the requirements, for courses that are not bar tested and may have alternate assessment schemes. Students who do well in their first year can then go on to choose courses that allow them to keep up or substantially improve their already good GPAs.[6] Students flagged for warning or probation after the first year have a much harder time moving up in class rankings in subsequent years. Students in academic difficulty know that on-campus recruiting is not going to even consider them. Clinical opportunities may also be lost because of scheduling or because of academic status or both.  Some students really need to take the engine apart and put it back together to understand how it works-and some students need to see what lawyering really is to reignite their underlying enthusiasm for continuing in law school. There is a hopelessness we are creating because these opportunities are lost.

Don’t get me wrong, I am not advocating that we abandon this process altogether. We do students a grave disservice if they are misled throughout law school to believe that they are on track for bar passage only to fail. We similarly do students no favors by continuing to take their tuition money when law school is clearly not for them. Perhaps, though, we can re-evaluate our methods. There are no easy answers here-just a request to be mindful of students who feel that they are drifting further away even as we are throwing them a lifesaver. They don’t want to just survive; they want the opportunity to get back on the ship.

(Liz Stillman)

 

[1] Carson v. Makin, 979 F.3d 21 (1st. Cir. 2020)

[2] Me. Stat. tit. 20–A, § 2(1) (2022).

[3] Carson, 979 F.3d at 26.

[4] Id.

[5] Id.  at 30 (citing Eulitt v. Me. Dept. of Educ., 386 F.3d 344 (1st. Cir. 2004)).

How’s my citation? Call 1-800-Bluebook to report it.

[6] Is this ideal for bar passage? Perhaps not.

April 25, 2022 in Learning Styles, Miscellany, Program Evaluation | Permalink | Comments (0)

Sunday, October 31, 2021

New Perspectives

Raise your hand if you told your child to do something, they ignored you, and then 2 days later they thought someone else was brilliant for telling them the same thing (go ahead, raising your hand can be therapeutic).  Raise your hand if you provided a piece of advice to a law student, they didn't fully buy in, and then they "discovered" the same piece of advice later that semester from someone else. 

Most of us probably don't have another hand to hold up, so I will stop there.  I don't think ignoring our advice is malicious or failing to trust the speaker.  Sometimes, people need more persuasion to make changes.  Sometimes, a different way of conveying the same information helps people.  Either way, new or different perspectives help.

New or different perspectives help ASPers as well.  The regional ASP conferences are starting with registrations and calls for proposals.  I encourage everyone to think of a proposal and submit at least 1 this year.  You are doing amazing things in the classroom and individually with students.  Share that with the rest of us.  I understand many people worry they aren't doing unique things.  First, don't sell yourself short because you are creating unique experiences for students.  Also,  you may be able to provide a perspective others haven't seen or explain a different way to teach something that would help others.  Our community continually improves as we share ideas, activities, perspectives, and challenges together.  You can help contribute to that progress.  

We tell our students to stretch beyond their comfort zone.  I encourage many in ASP to stretch as well.  It will help you and many of us improve student experiences.

(Steven Foster)

October 31, 2021 in Program Evaluation, Teaching Tips | Permalink | Comments (0)

Wednesday, October 13, 2021

Bar Exam Pass Rates and Academic Support

Maya Angelou wrote “we are more alike, my friends, than we are unalike.”  One of my favorite songs right now is Bleed the Same by Mandisa where she conveys a similar message.  I believe the message from both of them would apply to the current discussion surrounding factors impacting bar passage rates. 

Most of you are aware Rory Bahadur wrote a series of articles examining the relationship between certain factors and bar passage rates.  He specifically questions whether FIU’s emergence as the leader in Florida’s bar pass rate is significantly impacted by factors such as involuntary attrition, incoming transfers, and incoming credentials.  An oversimplification of his conclusion is that these factors have a major impact on Florida’s bar pass rankings.  His 3 articles are on SSRN here:

  1. Blinded by Science? A reexamination of the Bar Ninja and Silver Bullet Bar Program Cryptics
  2. Reexamining Relative Bar Performance as a Function of Non-Linearity, Heteroscedasticity, and a New Independent Variable
  3. Quantifying the Impact of Matriculant Credentials & Academic Attrition Rates on Bar Exam Success at Individual Schools

FIU’s academic support team, which includes one of our editors Louis Schulze, responded last weekend in a series of blog posts.  You can read the posts here:

  1. Does Academic Support Matter? A Brief, Preliminary Response to Blinded by Science and its Progeny
  2. Does Academic Support Matter? A Brief, Preliminary Response to Blinded by Science and its Progeny, Part 2

Louis’ response questions the statistical methods used in the previous articles and posits that FIU’s new Academic Support program made a statistically significant effect on bar passage rates.  Rory responded to the posts with a message on the ASP listsev/google group.  You should be able to access his message within that group. 

Rory and Louis are engaged in a relevant and important discussion for ASP.  I encourage everyone to read the articles and posts.  AccessLex also published a brief post addressing this topic and one of Rory’s articles.  The AccessLex authors state they are conducting  a couple projects that will provide even more insight.

The academic debate surrounding this topic is necessary, but we should also recognize the reason why the debate is important and sometimes personal.  While they disagree, both Rory and Louis are passionate about helping ASPers and students.  They both cite the lack of tenure for ASPers as a major concern.  They both argue for more resources for Academic Support.  Knowing them both, I truly believe they are trying to do what is best for both ASP and students.

As long as we are trying to figure out what helps students succeed, I do want this discussion to continue in an academic manner.  One of my major concerns is when schools/Deans evaluate whether ASPers are effective based primarily on bar pass rates.  Bar pass rates are an easy number to stamp on a department, almost treating bar pass numbers as wins and losses.  Media and other entities fuel that perception with articles about who had the highest bar pass rate in the state.  FIU’s success has brought national attention from the ABA journal and other legal news sources.  Deans around the country, especially ones in Florida, do specifically ask, “why isn’t [insert school] having the success of FIU?  Are our people doing their job correctly?”  Those outside ASP want to know, what is the secret sauce?

I also want the discussion to continue to demonstrate the impact ASP has on students.  Both Louis/Raul and Rory presented at regional and national ASP conferences about best practices in teaching.  Many of us agree that law school education and pedagogy needs improving.  Most of us agree that better teaching would improve student learning and that we should use scientifically proven methods to teach students.  We would also agree that improved student learning should have an impact on student success and bar performance.  I want to know what everyone else does, including Louis and Raul, to lead to improved student performance.  I especially want to read studies that quantify the impact of Academic Support and/or specific Academic Support programs.  Anecdotally, we know we have an impact on individual lives.  That impact matters, and should be measurable. 

Promoting ASP is important to the majority of us.  We need ongoing projects to measure what works and how we can all improve our students’ chances to pass the bar exam.  I know we are all striving to promote each other and help students.   I hope we can continue to do that.

(Steven Foster)

October 13, 2021 in Academic Support Spotlight, Bar Exam Issues, Program Evaluation | Permalink | Comments (0)

Tuesday, January 19, 2021

Academic and Bar Support Scholarship Spotlight

ASP Foundational Scholarship Series:  This series focuses on the seminal ASP/ Bar Exam scholarship that contributed to the development of academic and bar support best practices.  

For the first-ever post in this series, I was stuck between two choices.  So, I chose both:  

1.    Knaplund & Sanders, The Art and Science of Academic Support, 45 J. Legal Educ. 157 (1995). 

This article was one of the earliest and most robust empirical analyses of law school academic support programs.  It helped ASP faculty defend the then-controversial pedagogy of "contextualized academic support" and answer the question "Why should we spend money on an ASP?"

From the introduction:

• Our analysis of seven distinct academic support initiatives at UCLA shows that support can substantially and demonstrably improve both short-term and long-term academic performance, but the effects vary markedly across UCLA's programs.

• The variation in academic effectiveness across UCLA's programs follows distinct patterns that yield definite guidance on the pedagogy of academic support.

• We found some evidence that academic support programs can have valuable benefits apart from their impact on grades.

2.     Russell McClain, Helping Our Students Reach Their Full Potential: The Insidious Consequences of Ignoring Stereotype Threat, 17 Rutgers Race & L. Rev. 1 (2016).

Coupled with Professor McClain's conference presentations on this subject and a related TEDx Talk, this article was the first to analyze the phenomenon of stereotype threat specifically as it pertains to law students.  It serves as a crucial resource for ASP faculty, and all others, to understand their potential in ameliorating the effects of implicit bias in the law school classroom.

From the article abstract:

A psychological phenomenon may be a significant cause of academic underachievement by minorities in law school. This phenomenon, called stereotype threat, occurs as a result of the fear of confirming a negative group stereotype....  When subject to this threat — as a consequence of being confronted with environmental or explicit triggers — people do worse in academic settings than they otherwise are capable of doing. In this article, I explore the implications of the research on stereotype threat for law schools and make several recommendations to deal with the threat.

There are natural implications for law school admissions, of course. If a portion of our applicant pool is affected by stereotype threat, then we cannot trust the accuracy of the metrics we typically use in law school admissions, i.e., prior academic performance and LSAT scores of law school applicants. Indeed, those credentials actually may under-evaluate the academic potential of these applicants, who are often minority students. This should cause law schools to reevaluate their admissions policies.

After students are admitted, law school provides fertile ground within which stereotype threat can flourish. This, of course, means that the performance of minorities in law school — in class, on exams, and in other areas — is likely to be diminished, such that many minorities will not perform up to their academic capacity. And, obviously, we would expect this same dynamic to play out on the bar exam.

Law schools can address stereotype threat at each of these levels, and they should do so. This article lays out a framework for understanding and dealing with the threat.

(Louis N. Schulze, Jr., FIU Law).

 

 

January 19, 2021 in Diversity Issues, Program Evaluation, Publishing, Reading, Teaching Tips, Writing | Permalink | Comments (0)

Monday, May 25, 2020

Ask an Expert

Ordinary greatness. You’ll find it in places you never imagined and, when you do, higher performance and increased productivity are not far away. – Barbara Pagano

Last fall, a presenter at a workshop that I attended referred to the audience members as SMEs (pronounced smēz). She readily explained that SME = subject matter expert, and that those in attendance, based on our training and experience, were SMEs. As you can tell, I was tickled by the title. Before that workshop, I had not yet recognized my experience as expertise. During the workshop, and many times since, I told my imposter syndrome to kick rocks, and came to realize that the presenter was right.

Weekly, I write to and about the subject matter experts in every law school, whose expertise is designing and delivering academic enhancement and bar support programming. As my thoughts turn to this group of specialized experts, I ponder the number of meetings held, decisions made, funding resolutions, and curricular adjustments approved without any input or involvement from an ASP expert. Too often ASP expertise is overlooked, or called upon only in reaction to a problem that could have been proactively addressed, like fluctuating bar pass rates, and serving at-risk students.

In the turmoil of a global pandemic, plans for the fall semester are uncertain. The July bar exam is a hot mess topic right now. Deans, students, faculty members, examiners, and bar administrators are not sure of what the summer and fall will bring or what their next move will be. While ASP professors can’t solve any of the problems that COVID-19 has wrought, we most likely have greater insight into the learning needs of law students. Implementing decisions that affect the delivery of the program of legal education without consulting those with subject matter expertise in educational delivery, is like buying a house without a realtor. It can be done, but rarely without regret.

If we were to survey U.S. law schools, I wonder which would have ASP professors serving on or consulting with admissions committees that screen applicants to identify those most likely to successfully complete the academic program and pass a bar exam. How many law schools have a member of the ASP team on the curriculum committee where decisions that affect bar preparedness are made?

While student success is the primary concern of academic support programs, a well-structured ASP team will also complement a faculty of doctrinal experts and provide helpful teaching resources and opportunities for collaboration. It might surprise those outside of ASP to learn that experts in ASP are typically the first to know which students are struggling academically. Many students will seek out academic support assistance when they feel lost. Some, but certainly not all, professors may not become aware of a struggling student until after the one summative assessment is graded. Deans, directors, chairs, and coordinators who want to identify areas and sources of underperformance in the first-year and required curriculum could save time and resources by asking their in-house ASP experts first.

(Marsha Griggs)

May 25, 2020 in Miscellany, Program Evaluation | Permalink | Comments (0)

Tuesday, October 22, 2019

The Charisma of Numbers

Today's Washington Post has a fascinating and disturbing article about the company HireVue and its signature product, an artificial intelligence hiring system through which employers can set up automated "interviews" with prospective employees.  The system "uses candidates’ computer or cellphone cameras to analyze their facial movements, word choice and speaking voice before ranking them against other applicants based on an automatically generated 'employability' score."  Based on these scores, HireVue's clients -- which include large organizations like Unilever and Goldman Sachs -- can choose which candidates they would like to bring in for actual human interaction.

The growing reliance of employers on HireVue and its competitors suggests several issues of interest to law students.  Can we expect that someday soon, they too will be forced to welcome their new computer overlords by developing another set of skills -- namely, the art of using just the right expressions and intonations to appeal to the interviewing algorithm?  How do we even know what appeals to that algorithm, and whether the appealing features actually bear any relationship to job performance, if HireVue releases no information about what it is measuring, what it assigns value to, or, indeed, even what a candidate did wrong?  (The mystery and validity issues echo some complaints about the UBE, but at least bar examinees are told their scores.)  Like it or not, this Pandora's boxing ring is now open, and it's only a matter of time until young attorneys are sent in to altercate.

To get some perspective on the rigor of the HireVue system, the Post reporter spoke to researchers in applicable fields, including Luke Stark, an AI researcher who was

skeptical of HireVue’s ability to predict a worker’s personality from their intonations and turns of phrase. . . . Systems like HireVue, he said, have become quite skilled at spitting out data points that seem convincing, even when they’re not backed by science. And he finds this “charisma of numbers” really troubling because of the overconfidence employers might lend them while seeking to decide the path of applicants’ careers.

The charisma of numbers is something I feel I run up against over and over again.  And I say this as a person who values data and statistics!  I believe it is difficult to make consistently effective decisions or to take wise action without obtaining and evaluating relevant numerical information.  And, true, in a field in which our success is largely measured numerically (GPAs, retention rates, bar passage rates), numbers can possess either star power or infamy.

But, notwithstanding their dazzle and clout, numbers should only be powerful if they are attached to something meaningful.  If they are being misused or misunderstood, that can mean mistaking the sizzle for the steak.   Figures can be seductive when they seem rounded, or extravagant, or provocative, or revealing.  It's easy to jump on the conspicuously appealing numbers -- the highest GPA, the apparently significant pattern in MBE scores, the increase in median starting salaries -- just as it's easy to be attracted to the confident, well-spoken cutie who walks into the party.  But the GPA might be based on a disproportionate number of generously graded courses; the MBE pattern might be statistically insignificant; the median salary increase might represent slippage, not advancement, if similar schools are seeing an even larger increase.  Causes, reliability, and context all matter.

The danger of the charisma of numbers is that sometimes, even when a person is only looking at the surface, they don't feel like they are being shallow, because numbers are supposed to be scientific and rational.  We need to remember, and teach our students and colleagues, that, even with the most alluring numbers, you should really spend some time with them first, get to know their flaws and idiosyncrasies, before you commit to them.

[Bill MacDonald]

October 22, 2019 in Advice, Bar Exam Preparation, Current Affairs, Diversity Issues, News, Program Evaluation, Science | Permalink | Comments (0)

Saturday, August 17, 2019

WCCASP Accepting Proposals

The West Coast Consortium of Academic Support Professionals is hosting their 8th Annual Conference on November 1st in Las Vegas.  The theme is Technology and Data Assisted Academic Support Programming.  They are accepting proposals through September 23rd.  I attached an image of the flyer they sent out below.

WCCASP

August 17, 2019 in Program Evaluation, Teaching Tips | Permalink | Comments (0)

Sunday, July 28, 2019

Deep Work

The sun is shining through my windows.  The day is starting well.  I turn on my computer, and outlook starts.  The barrage of emails then piles into my inbox.  I methodically answer questions about internships, class selection, the bar exam, exam taking, and other student concerns.  As I finish my hour slog through email, my first student appointment comes in.  We talk about life and law school.  I finish the meeting and plan to work on my upcoming class.  After pulling up the class schedule, I start planning the next class period.  My phone vibrates (which also vibrates my watch).  I get a text message about the half-price deals at Top Golf this week.  I get back to thinking about my next class when my phone vibrates again.  This time a group text about a kids activity.  9 messages later, I am back to thinking about class.  Out of the corner of my eye, I see that my email box now has 12 emails.  3 of them can be immediately deleted, and 3 can be answered in 1 sentence.  The rest take more time, so I will let them sit for now.  Back to working on class, and a student needs to see me.  By the end of the day, my class exercise still isn't done.  Maybe tomorrow, or I may just do the same thing as last year.

Am I the only one that has this experience?  Probably not.  I am sure everyone has similar problems.  Technology is infiltrating every second of my day, so my day includes tons of small breaks.  The breaks cause me to lose focus, which in turn means I take longer to accomplish any task.  

The book Deep Work by Cal Newport addresses this topic.  He states that we are all so inundated with constant beeps and buzzes that our brains are being trained for only small tasks.  We are addicted to the immediate need to respond or help that we can't accomplish more difficult and meaningful tasks.  Deep work is when someone takes long uninterrupted time to contemplate a project.  Hours of preparation and thought to innovate.  Hours?  I hope I get double-digit uninterrupted minutes to work on a task.  His book is great at illustrating the problems with the constant interruptions and how societal breakthroughs are more difficult without high levels of focus.

Newport advocates for everyone to engage in deep work.  He concedes that most people can't spend 4-5 uninterrupted hours on a project every day.  However, we can only look at email at specific times.  Creating routines that eliminate distractions for set periods of time can help.  The goal is to not let anyone intrude on deep work time to enable quality thinking.  The more time, the more we can accomplish.

Our students could use this information as well.  I went to law school prior to smart phones, so I didn't have that distraction.  However, the internet was enticing.  One of the best things I did in law school was not connect my computer to the wifi my first 2 years.  When I was studying, I couldn't check email, browse facebook, or do anything online.  I read and briefed cases.  I would check email when I got home.  My focus during law school was probably my best focus ever.  Most people don't have the willpower to not use technology.  Putting phones in the other room or turning everything off is good.  Eliminate distractions to improve focus.

I am working on trying to block out more time during the day for focus.  Finding the time is difficult, but if I want to continually improve my programming, I need the time to think about it.  Just like I tell my students, I must intentionally create quality time.

(Steven Foster)

July 28, 2019 in Program Evaluation, Study Tips - General | Permalink | Comments (0)

Sunday, July 7, 2019

Fall Planning

The calendar turned to July.  The heat index is climbing over 100 in Oklahoma, which also means it is time to plan the fall semester.

The difficult part of July is everything happens at the same time.  The bar exam is still 3 weeks away, but I can’t wait until after the bar to plan for the fall.  The semester will start only a couple weeks later, and in an attempt to follow my own advice, I take time off right after the exam.  The timing means fall planning must happen now.

I am sure many in ASP have a similar timeframe.  When thinking about the fall, I wanted to pass along a few tips:

  1. Look back at the AASE materials. AASE presenters provide numerous ideas.  Implementing the great ideas is the more difficult part, but one way to increase the chance of implementing them is to re-read those ideas now. 
  2. Review feedback from last year. If students provide constructive feedback, then look through the feedback prior to creating syllabi or programming.  Small changes each year can make a big difference.  However, ignore any non-constructive criticism. 
  3. Analyze notes and materials from last year. I suggest writing down what works and doesn’t work throughout the year.  Look at those notes to see what changes are needed.
  4. Decide on a few new things to integrate. Don’t try to remake the entire ASP program in one semester/year. 

The summer is winding down, fortunately or unfortunately depending on who you talk to.  Many professors are planning for the fall.  Spending the few extra minutes looking at great ideas, feedback, and notes can gradually improve programming.

(Steven Foster)

July 7, 2019 in Program Evaluation | Permalink | Comments (0)