Development of a general solution to the problem of validity generalization.

Abstract

U. S. Civil Service Commission andGeorge Washington UniversityJohn E. HunterMichigan State UniversityPersonnel psychologists have traditionally believed that employment test valid-ities are situation specific. This study presents a Bayesian statistical modelwhich allows one to explore the alternate hypothesis that variation in validityoutcomes from study to study for similar jobs and tests is artifactual in nature.Certain outcomes using this model permit validity generalization to new settingswithout carrying out a validation study of any kind. Where such generalizationis not justified, the procedure provides an improved method of data analysisand decision making for the necessary situational validity study. Application tofour distributions of empirical validity coefficients demonstrated the power ofthe model.A recent study (Schmidt, Hunter, & Urry,1976) addressed the belief, dominant in per-sonnel psychology, that meaningful empiricalvalidation studies are possible for most, if notall, jobs in most organizations. This studyshowed that, because of range restriction andless than perfect criterion reliability, thesample sizes necessary to provide adequatestatistical power are usually much larger thanhas typically been assumed, This finding leadsto the conclusion that empirical validitystudies are technically feasible much lessfrequently than the profession has assumed.The present study is addressed to another ofthe orthodoxies of personnel psychology: thebelief that test validity is generally highlysituation specific. Considerable variabilityfrom study to study is observed in raw valida-tion results even when jobs and tests appearto be similar or essentially identical (Ghiselli,1966). The explanation usually advanced forthis phenomenon is that the factor structureof job performance is different from job tojob and that the human observer or job ana-lyst is simply too poor an information receiver

Keywords

Related Publications