CrimProf Blog

Editor: Kevin Cole
Univ. of San Diego School of Law

Monday, August 3, 2020

Graham et al. on Samples for Studying Criminal Justice

Amanda GrahamJustin Pickett and Frank Cullen (Georgia Southern University, School of Criminal Justice, University at Albany, SUNY and University of Cincinnati) have posted Advantages of Matched Over Unmatched Opt-in Samples for Studying Criminal Justice Attitudes: A Research Note on SSRN. Here is the abstract:
PURPOSE: Despite the growing popularity of online opt-in samples in criminology, recent work shows that resultant findings often do not generalize. Not all opt-in samples are alike, however, and matching may improve data quality. Replicating and extending prior work, we compare the generalizability of relational inferences from unmatched and matched opt-in samples.

METHODS: Estimating identical models for four criminal justice outcomes, we compare multivariate regression results from national matched (YouGov) and unmatched (MTurk) opt-in samples to those from the General Social Survey (GSS).

RESULTS: YouGov coefficients are almost always in the same direction as GSS coefficients, especially when statistically significant, and are mostly of a similar magnitude; less than 10% of the YouGov and GSS coefficients differ significantly. By contrast, MTurk coefficients are more likely to be in the wrong direction, more likely to be much larger or smaller, and are over twice as likely to differ significantly from GSS coefficients.

CONCLUSIONS: Matched opt-in samples provide a relatively inexpensive data source for criminal justice researchers, compared to probability samples, and also appear to carry a smaller generalizability penalty than unmatched samples. Our study suggests relational inferences from matched opt-in samples are more likely to generalize than those from unmatched samples.

| Permalink


Post a comment