Forming Composite Scales and Estimating Their Validity Through Factor Analysis

Abstract

For forming a scale through factor analysis and evaluating it with Heise and Bohrnstedt's validity rho, the factor analysis should include the indicators of only one construct. If these variables are optimally weighted through least-squares regression, rho is the multiple correlation of the factor with the measured variables. An often tenuous assumption underlies the use of rho to correct a scale's correlations for invalidity and error. Although useful, rho is not a substitute for external validation. Maximizing rho is a better criterion for selecting weights of variables than is maximizing Heise and Bohrnstedt's reliability index. Although alternative measurement approaches have received considerable attention recently, estimating the values of underlying constructs with additive composite scales is still an important tool in social and psychological research. Frequently, data from one sample at only one or two times are all a researcher has for selecting items and their weights and for evaluating the reliability and validity of the composite. In such circumstances, one must rely in great part upon internal-consistency estimates of reliability and, to a lesser extent, upon evidence of validity to the set of items selected for the composite. For many years there have been several such indices of reliability, but there has been no generally known internal index relevant to the widely accepted definition of validity as the accuracy of measuring a particular construct distinguished from other constructs and method factors (Campbell and Fiske, 1959). In 1970, Heise and Bohrnstedt made two major contributions to the tools for evaluating composite scales: first, a generally more accurate internal-consistency of reliability than those previously available and, second, a relevant and easily calculated and interpreted of validity. However, they did not develop some important strengths and weaknesses of their indices. The strengths increase their utility, while the weaknesses remind us of the caution with which we must use even the best procedures for estimating theoretical values. In this article, we shall discuss Heise and Bohrnstedt's validity rho (pfc) and some related aspects of composite scales formed through factor analysis (for an evaluation of their reliability index, see [Smith, in press]). The article is divided into three main sections. 1. To make our algebra less cumbersome, we shall make some nonessential simplifying assumptions and adopt a vector representation of individual values. Then as a foundation of what follows, we shall briefly review the factor model and its relation to composite scales. 2. We shall describe an optimal weighting method which produces higher reliabilities and validities than the weighting procedures frequently employed. Moreover, with these optimal weights, pf, has some very important additional interpretations. 3. From the perspective developed in the first two sections, we shall make some brief observations about (a) the estimation of correlations with underlying constructs, (b) the limitations of estimates of validity, and (c) the choice between maximizing reliability or validity indices as a scaling criterion. Although familiarity with the original Heise and Bohrnstedt article is probably not essential for understanding the present one, it no doubt would help. THE FACTOR MODEL AND FORMING A

Keywords

Affiliated Institutions

• Princeton University US

Related Publications