TortsProf Blog

Editor: Christopher J. Robinette
Widener Commonwealth Law School

Monday, June 14, 2010

Your Input Needed On Law Review Survey

Lucy McGough (LSU), James Bowers (LSU), and Richard Wise (UND - Psychology) are trying to assess the current system of student-run law reviews:

For the last several decades, there has been much controversy and discussion about how well the current system of student run law reviews and journals meet the needs of legal scholars, the legal profession, and its student members and how they can be improved. Despite the significance of this controversy, no one has determined the legal community's opinions about them.

The purpose of the present survey is to assess: (1) What law professors, attorneys, judges, and law  review editors think about the current system of student run law reviews and journals; (2) Whether reforms are needed; and (3) If reforms are needed, what they should be.

We are asking for your help with this important survey because the success of any survey depends in large part on the number of people who complete the survey. The present survey is completely anonymous and confidential and only takes about 15-20 minutes to complete. The results of the survey will be reported in a law review article. A link for the survey is enclosed below: 

Thank you very much for considering our request.



TortsProfs | Permalink

TrackBack URL for this entry:

Listed below are links to weblogs that reference Your Input Needed On Law Review Survey:


I did the survey, but I have some concerns and a question about the design of it.

First, and most significantly, although the survey claims to be "completely anonymous", it isn't in fact meaningfully anonymous at least as regards law faculty: asking (1) where we teach, (2) our tenure status, (3) whether we were law journal editors and if so (4) what type, and (5) how many articles we have published, plus (6) what year we graduated from law school, and (7) from what law school -- provides amply sufficient info to distinguish law faculty members individually. I'm pretty sure I'm the only 1987 Yale law grad who was a law review editor and has published 28 law review papers at the University of Miami. In fact, I'm pretty sure I'm the only 1987 law grad. I am certain the investigators are above reproach, and the subject isn't a very sensitive one for most people, but I wish people wouldn't equate "not asking your name" with "completely anonymous" as they are not at the same thing.

Second, since I'm quibbling anyway, when asked where we have published, I for one didn't understand the distinction between a "student edited law review" and a "student edited journal" (is it what they call themselves, or something substantive? If the latter, what?)

Third, the question about whether there was the wrong level of faculty involvement in law reviews was very ambiguous. Some may feel it is too high; others may feel it is too low -- I fear that the survey authors will happily lump both groups together -- maybe as pro-faculty involvement? And then there are people like me who think in some schools it is too high and others too low -- how do I answer this one?

And finally, can a self-selected survey-monkey sample adequately substitute for a truly randomized survey, with a randomly selected population, regardless of the online response rate? Has anyone tested this more recently than this 1999 Pew Research paper? (Internet penetration is now much greater so I think we need fresher data before just accepting the negative result.) If not, if there is survey-type induced bias, what if anything will the results mean?

None of this is meant to suggest that the issue of law review management isn't an important one -- although I would be prepared to argue that post-publication reviews like Jotwell help overcome some of the inherent flaws of the current system by drawing attention to what experts consider to be the 'good stuff'.

Posted by: Michael Froomkin | Jun 14, 2010 12:44:51 PM

Response to Professor Froomkin’s Comments:

The only way that some respondents could be identified is if we examined all their demographic information and then attempted to identify them from their websites, which we will not do. First, when we obtained IRB approval, we promised to keep all respondents’ information confidential. Second, there is no reason identify individual respondents. The responses to the survey will be analyzed by groups (i.e., law professors, law review editors, attorneys, and judges). We have no interest in how an individual responded to the survey unless he or she made an anonymous comment that he or she intended us to read. What is important is how the different groups of respondents perceive the strengths and weaknesses of law reviews not how an individual respondent perceives them. Thus far we have received a good response to our survey with 846 law professors, 121 law reviews editors, 163 attorneys, and 75 judges having taken the survey. Attempting to determine how over 1200 respondents individually answered the survey would be a fruitless and time consuming task.

Furthermore, the demographic information we requested will help us to analyze the data. For example, does the number of law review articles published by a law professor affect his or her view of law reviews? Do law professors who were former law review editors perceive law reviews differently than law professors who were not law review editors? Do law professors’ opinions of law reviews vary depending on their law school?

We intended student edited law review to refer to a law school’s general law review while a student edited journal refers to any other type of law school publication.

We asked in the survey for respondents to agree or disagree with the following statement: “The level of faculty oversight hurts the ability of law reviews to publish the best articles.” Certainly, you are right when you say that a respondent’s answer may depend on the law review and the law school. Nonetheless, we believe this question help us to address an important issue: Namely whether the different groups of respondents believe that in general greater law faculty oversight is necessary to improve the quality of law reviews.

You are also correct that obtaining a representative sample would be preferable, but obtaining such a sample would be very difficulty for several reasons. First, we would need not just a representative sample of law professors but also a representative sample of law review editors, judges, and attorneys. Such an endeavor would be very time consuming and expensive and beyond our resources. Second, persons who complete a survey tend to be more interested and more knowledgeable than persons who do not complete a survey, which would add to the difficulty of obtaining a representative sample for each of the different groups of respondents. Third the more responses we receive the more likely our samples are to be representative of their respective populations. Lastly, many surveys are conducted without representative samples, but nonetheless provide useful information and are published in peer reviewed journals. For example, I have published the results of several surveys in peer reviewed journals though they did not have representative samples. See e.g., Richard A. Wise & Martin A. Safer, What U.S. Judges Know and Believe about Eyewitness Testimony, 18 APPLIED COGNITIVE PSYCHOL. 427 (2004); Richard A. Wise et al., What US Prosecutors and Defense Attorneys Know and Believe about Eyewitness Testimony, 23 APPLIED COGNITIVE PSYCHOL. 1266 (2009); Richard A. Wise & Martin A. Safer, A Comparison of What U.S. Judges and Students Know and Believe About Eyewitness Testimony, J. APPLIED SOC. PSYCHOL. (forthcoming); Richard Wise et al., What U.S. Law Enforcement Officers Know and Believe about Eyewitness Factors, Eyewitness Interviews, and Identification Procedures, APPLIED COGNITIVE PSYCHOL. (forthcoming). The first of these publications has been cited 28 times.

Finally, we want to thank everyone who has taken our survey! We sincerely appreciate your time, effort, and insightful comments!

Posted by: Richard Wise | Jun 21, 2010 5:02:39 PM

Post a comment