Test Everything

Now that Thanksgiving is past, most law students have started or are about to start preparing for final exams.  With an entire semester of material to master, many try to prioritize what to spend their time on.  Some concepts and rules, introduced early in the course, may feel tediously familiar through repetition, and students may feel they can afford not to spend time on them now, especially if they tested successfully on those points on the midterm.  Experimental evidence suggests, however, that time could still be well spent, even on familiar material, if it is spent the right way.

Jeffrey D. Karpicke and Henry L. Roediger III, psychologists at Washington University, ran a series of tests of memory recall of lists of words, examining the effects of two distinct tasks that contribute to learning: studying and testing.¹  Their first set of tests, similar to other experiments that had been done over the previous 40 years, sought to determine what combination of studying (in this case, visually reviewing lists of words to be memorized) and testing (writing down recalled words under time pressure) would produce the best learning.  Some subjects were told to study a list of words, then were tested on their recall, then given another opportunity to study, and finally given one last test.  These were identified as STST subjects.  Other subjects were given disproportionate opportunities to study, or to test: either three study periods and one test (SSST) or one study period followed by three tests (STTT).  The scientists then compared recall performance for different groups of subjects after the final test. Hopefully not surprising to either law students or Academic Success professionals, the subjects in the STST group had the best recall in the end.  As other psychologists have observed, a mix of studying and testing produces the best learning.

What was new and interesting was the second phase of their testing, in which they variations on the STST pattern on new groups of subjects, and tested recall not just at the end of the four-step pattern, but also on an extra test given one week later.  In addition to testing some volunteers using the original STST method (study, test, study, test), and to tweaking the order (but not the ratio) by giving some volunteers two study sessions followed by two test sessions (SSTT), the scientists tested a third set of volunteers by starting with the same size set of words in the first two steps (study, test), but then removing from the word list all the words successfully recalled at that point, and then asking the volunteers to study and then test using only the words not recalled in the first test.  Using the reduced list of words was identified with a subscript "N", so this set of examinees was called "STS_NT_N".  [This method is familiar to many who study (or recommend studying) with flash cards by removing from the deck each card you recall correctly, so that every new pass through the deck, you are only studying and testing yourself on the information you failed to recall the last time.]  Finally, a fourth set of volunteers similarly started with a complete list of words, which they studied and were tested upon, and then spent a second study period studying only the reduced list (that is, again, they did not have to study any words they had already learned).  However, on their second test, all of the words from the initial list were tested, even those that were not studied a second time.  Thus, this was the "STS_NT" group.

What Karpicke and Roediger discovered this time was that the STS_NT_N group clearly had the best recall after the second test -- in fact, among the fifteen subjects tested in this group, there was only a single instance of a word not being recalled.  In other words, this group had the "fastest initial learning" of all, apparently because they focused both their studying and their testing on material that they had not previously learned.  They learned the material more quickly than either group that studied and tested on all the words twice (STST and SSTT) or the group that studied only material that it had not previously learned, but tested on all the words twice (STS_NT).  And this has an intuitive appeal to those of us who have used flash cards -- if you focus on your gaps, you can overcome them more quickly, right?

However, the scientists also tested all the subjects one week later, asking them to recall all the words from their initial lists.  And on this later test of long-term memory, the STS_NT_N group performed the worst.  One might simply attribute this to those subjects having spent less total time and mental energy on the learning task, since they had a reduced word list for the second half of their learning.  If that were the case, though, you would expect that the best long-term results would have been seen in the STST group, which studied and tested on full word lists twice.  In fact, the long-term performance of the STS_NT group was as good as, and in some cases better than, the performance of the STST group.  This suggests that it does not matter so much whether you spend time studying material you already know, as long as make sure to continue to test yourself on that material along with the material you are continuing to learn.

So over the next few weeks, as our students work somewhere between diligently and frantically to prepare themselves for their final exams, it seems that the most efficient and effective use of their time will be to focus their study time (reviewing, rote memorization, consulting supplementary material, asking TAs and professors questions) on the things they are unsure of, but to continue to test themselves on everything in the syllabus.

¹Jeffrey D. Karpicke & Henry L. Roediger III, Repeated retrieval during learning is the key to long-term retention, Journal of Memory and Language 57 (2007) 151-162.

[Bill MacDonald]