Project: A Comparison of Relational Validity Analyses with Simulated Behavioral Data

The purpose of this study was to evaluate which various methods accurately estimate the true within-classroom and between-classroom correlations when both scores contain a rater or contextual effect. In this study, we simulated data to closely mimic behavioral assessment data as could be expected if a researcher were to administer both a brief behavioral screener and extended behavioral assessment. With each simulated data set, we estimated the relation between the scores on each instrument using multiple methods (correlations, multilevel linear regression, structural equation models [SEM], and multilevel SEM) and compared each estimate to the true relation set in the simulation parameters to determine which method(s) most accurately captured the true relation between screening and criterion assessment scores.