P S Y C H O M E T R I K A - - V O L . 11, NO.

3
SEPTI~MBER, ]~4,6
A N A P P R O A C H TO T H E P R O B L E M OF
D I F F E R E N T I A L PREDICTION
H U B E R T E . B R O G D E N
T H E A D J U T A N T GENERAL'S OFFICE*
A procedure for maximizing selective efficiency is developed for
application to situations in which it is desired to select from a single
group of applicants for several possible assignments. The problem
of comparable units for the several cri t eri a whose values must be
compared to each other for differential assignment purposes is dis-
cussed. I t is demonstrated that, assuming linear regressions, maxi-
mal selection is obtained if individuals in any given assignment are
differentiated from those rejected according to critical rejection
scores on the mult!ple weighted sum of the predictors and from an-
other passible assignment by critical difference scores which are
merely the differences between the wo critical rejection scores.
Since the relationships j ust indicated give no way of determining
the magnitude of the critical scores required to select the required
number of persons for each assignment, a successive approximation
procedure for accomplishing this purpose has been devised and a
computational example is worked out.
The procedures for obtaining maximum efficiency in selecting
personnel by means of test scores or other predictors are simple and
well known when a single assignment is involved. So far as the au-
thor is aware, no procedure has been devised for maximizing efficiency
of selection and assignment when each individual may be eligible for
several assignments. The present paper will be concerned with pre-
sentation of such a procedure.
Before attempting to formulate the problem in mathematical
terms, the question of comparability of the units for the criteria of
the several assignments will be given some consideration. While cri-
teria in standard score form might be regarded as comparable, this
solution involves the tacit and undesirable ~sumpt i on t hat all cri-
t eri a are equal in both variability and importance. In certain assign-
merits the nature of the work may be such that all individuals pro-
duce very nearly the same amount, while in other assignments con-
siderable variation may occur. It would, of course, be advantageous
to place an individual equally good at both t ypes of work in the lat-
ter assignment. Similarly, jobs vary in importance to over-all effici-
* The opinions expressed are those of the aut hor and are not to be construed
as official or as those of the War Department.
139
140 PSYCHOMETRIKA
ency of the organization. Thus, if an applicant for employment in a
newspaper office were highly but equally skillful at both sweeping
floors and operating a linotype machine; it would be desirable to have
him operate the linotype machine. Variation in the efficiency with
which the linotype machine is operated affects over-all efficiency of
the newspaper office much more than variation in the efficiency with
which floors are swept. The following discussion of the character-
istics of a meaningful criterion is pertinent to the problem of the
comparable units and leads rat her directly to the solution which seems
to the author to be most desirable. While it will be assumed in this
discussion of criteria ,that t he classification problem is t hat of an
i ndust ri al concern, most of the comments are with some modification
pertinent to classification problems of organizations such as the
United States Civil Service Commission or the Army.
Although standard scores were considered defective from the
viewpoint of comparability, it is definitely desirable to employ mean
deviates, and it will be assumed hereaft er t hat all variables are ex-
pressed in such terms. The only reasonable alternative to mean devi-
ates would be variables expressed in t erms of absolute zero. Apart
from the impracticability of at t empt i ng to determine absolute zeros
in our present stage of development of selection procedures, ,the in-
formation made available by determining absolute zero is not directly
pertinent to the problem of making the best possible selection and
assignment from the available applicants. In most selection problems
the essential comparison is between the given individual score and
the expected score if one were to choose at random from the sample
of applicants. This expected score is of course the mean. Selection
of an individual producing ten units more than the average applicant
would effect a saving of exactly ten units no ma,tter whet her the mean
were t went y above absolute zero or one hundred above it.
Usually in selection problems a test or a bat t ery of tests is em-
ployed to identify in a group of applicants those individuals who will
perform most efficiently on the job. Presumabl y if time and expense
were not important, the criterion itself would be employed as the
selector. Ideally, a criterion would indicate the difference between
the cost to the employer per unit of produce or per service rendered
by the given individual and the average applicant (t hat is, it would
be expressed in mean deviate form) multiplied' by the number of each
of the various types of units called for in the j ob t hat the given indi-
vidual could be expected to produce in a given time unit. The criterion
should allow for errors, training costs, turnover, or for any addition-
al cost accounting factors which might be related to individual dif-
ferences in the abilities or t rai t s of the persons on whom the criterion
HUBERT E. BROGDEN 14[
scores are obtained. For example, cert ai n t ypes of overhead would
be reduced wi t h more efficient producers in some instances. Thus the
cri t eri on would indicate t he total savi ng (or loss) to be obtained by
selecting the given i nst ead of t he average applicant. I f the cri t eri on
were expressed directly in t erms of a reliable measure of the savi ng
in dollars, the obtained cri t eri on scale would be in units havi ng the
same meani ng at all points of the scale. Ill addition, units of several
cri t eri a so expressed would have the same meani ng, or we mi ght say,
selective sio'nificance, and could be directly compared not only wi t h
ot her uni t s on the same cri t eri on scale, but wi t h any uni t on any cri-
t eri on scale. That is, i f individual X in Job A could be expected to
perform such t hat a savi ng of one hundred dollars would be expected
(over the average individual) and in Job B such t hat a savi ng of
two hundred dollars would be effected (the cri t eri on values would be
+100 and +200, respect i vel y), it could be said t hat a sav.ing of one
hundred dollars would be obtained by placing t he individual in Job B.
If, furt hermore, successive pairs of individuals were to have cri t eri on
values of --100 and --98, 105 and 107, and 95 and 97 m Job A ; and --22
and --20, 117 and 119, and 205 and 207 in Job B, t he differences in de-
si rabi l i t y :for empl oyment between these successive pairs would be ex-
actly the same as f ar as the employer is concerned. In t hat sense,
then, it can be said t hat t he cri t eri on units are comparable.
I t is realized t hat t he reasoni ng here is from t he employer' s view-
point only, and t hat t here are, even so, many intangibles t hat would
never be expressed in monet ar y terms. In fact, a close approxi mat i on
to such a criterion could probably not be obtained. However, consid-
erat i on of the desirable charact eri st i cs of t he cri t eri on may help in
obt ai ni ng the best approxi mat i on possible under t he given circum-
stances. The desi rabi l i t y of obt ai ni ng such an approxi mat i on would
be much more evident in a differential prediction problem t han in
those involving a single criterion, since, in t he former, the problem
of compari ng units of di fferent cri t eri a is added to t hat of compari ng
uni t s at di fferent part s of t he scale of t he same criterion. I nsof ar as
the units and scales are not comparable in t he way in which we have
defined comparability, t he selection procedure to be described will not
obtain maxi mal results. Very probably, it would be desirable to em-
ploy wei ght s det ermi ned by subjective j udgment as to the i mport ance
of t he job r at her t han to employ rat i ngs or ot her such cri t eri a in raw
or st andar d score form. However, it should possibly be emphasized
t hat t he form of file procedures to be developed here are not depen-
dent upon the charact er of t he units employed, even t hough the re-
sults themselves may be considerably affected.
I f we assume t hat our cri t eri a are expressed in t erms of dollars
142 P S Y C H O M E T R I K A
as units, the object of the selection procedure would be to maxi mi ze
the saving. Procedures for accomplishing thi s will be developed in
the fol l owi ng presentation. Let us assume timt:
1. All zero-order and partial regressions are linear.
2. All predicted criterion values have been computed f or all cri-
teria f or the same battery of tests. The symbol 9~ will refer
t o a predicted value of any given criterion i .
3. All statistical constants refer or apply to the sample of ap-
plicants.
It may be readily shown--simply, by summi ng the various arrays
- - t hat
where the subscript s indicates that summation is wi thi n those above
a poi nt of cut on the predictor. If the criterion is expressed in terms
of dollars saved, i t can be seen that ~, 9~ gives the amount saved f or
that selected group by the selection process.
With n assignments, the total savi ng in dollars (the cri teri on)
would be
I - - ~ , 9 ~ + ~, 9, + ' " + ~. ~, +- ' - + ~, ~, , (2)
I , the total savi ng i n dollars, is the index to be maximized.
Note: The reader is reminded that, i f no bi as ent er s into the prediction, the
algebraic sum of the errors of prediction approaches zero, so t hat :Z~ equals Zy.
Note though t hat a~ equals r r ~ j a ! , or, in t erms of multiple prediction a 7 equals
v- This l at t er point is significant We have already i ndi ca~d t hat
o U, if determined in proper units, would increase as the "importance" of the job
increases. Since i n practice we must employ test scores as predictors, Zy must
be calculated for those selected by the tests. This is equivalent to selecting on y.
Hence, the fact t hat a- is a function both of ay and the multiple correlation means
Y
t hat the accuracy with which the criterion can be predicted will have considerable
effect upon the differential assi gnment of the applicants, since use of y instead of
y values is equivalent to weighting according to the size of .the correlation. That
the weighting for purposes of differential prediction is properly a function of R
should, in any event, be apparent. I f the multiple correlation is zero for a given
assignment, the test scores are completely unrel at ed to the criterion values and
to the desirability of selecting individuals obt ai ni ng these scores. Hence, it is
apparent t hat the assi gnment involved should be completely disregarded in se-
lecting men for other assi gnment s even though it may be f ar more i mport ant to
obtain men high i n t hat assi gnment t han i n the case of any remai ni ng assign-
ments. While the significance of this general principle is most evident in differen-
t i al prediction, it has definite implications in employing a single predictor to
select for several assignments.
HUBERT E. BROGDEN 143
The problem is t hat of defining the bounding surfaces distin-
gnishing between the various assigned groups and differentiating the
assigned groups from the unassigned in order to obtain the desired
maximized value of I . While an exact general solution would involve
furt her and highly restrictive assumptions (i.e., normality of the cor-
relation surfaces) concerning the nat ure of the frequency functions
and would in any event be exceedingly complex, we shall be content
in the present paper with demonstrating t hat I will be maximized if
arbitrarily determined "critical rejection scores" on the ~ values are
employed to distinguish between assigned groups and those rejected
entirely, while the differences between all possible pairs of such criti-
cal rejection scores are employed as "critical difference scores" on
the "difference variables" to distinguish between the various possible
assignments. It will be noted t hat the proposed solution requires that
three separate propositions be demonstrated:
1. The desired bounding surface differentiating between the as-
signed group i and the rejected group is defined by a given
critical rejection score on 9~ (note the direct implication t hat
the bounding surface is not curvilinear).
2. The desired bounding surface between any two assigned
groups such as i and 3" is adequately defined by a critical dif-
ference score on the difference variable ( ~ -- ?~j).
3. The exact desired critical difference score on ( ~ -- Yi) is the
difference between the two critical rejection scores involved.
No equation for directly determining the exact critical rejection
scores is to be developed although a method of successive approxi-
mations will be suggested.
With reference to all of the propositions to be proved, it will be
helpful to note t hat it is axiomatic t hat I is maximized when (1) all
values for any rejected individual are equal to or lower than any
value for assigned individuals on the criterion of t hei r assignments
and (2) it is not possible to replace individuals who have higher 9
values on other t han the criterion of their assignment with unassigned
individuals such t hat the loss effected by replacement is smaller than
the gain effected by change in assignment, or it is not possible to effect
a gain by any combination of such replacements and reassignments.
Although the first of our propositions must be t rue if the assump-
tion of linear regression is met, this relationship between the assure 9-
tion and the proposition is not immediately evident. Suppose, i~.l a
multidimensional space with the ~'s as coordinates, we were to pt'e-
pare separate plots of individuals having successive ~3., scores ft' cm
144 PSYCHO]~IETRIKA
- - 3. 0 up t o +3. 0. That is, t he f i r st pl ot woul d consi st of all i ndi vi dual s
havi ng a ~ scor e of --3. 0, t he second of i ndi vi dual s havi ng a
~ scor e of --2. 9, etc. Ea c h of t hes e pl ot s woul d cons t i t ut e a
"sl i ce" t hr ough t he hype r s pa c e i n whi ch t he i ndi vi dual s we r e pl ot t ed.
I t is qui t e a ppa r e nt t ha t our r ej ect i on boundi ng s ur f ace, t ha t is, our
me a ns of di s t i ngui s hi ng bet ween t hose t o be r ej ect ed and t hose be-
l ongi ng in t he gi ven as s i gnment , mus t be one of t hes e "sl i ces, " si nce
wi t hi n each "sl i ce" all Y, val ues ar e t he s ame and wi t hi n t he "sl i ces"
above or bel ow all ~)~ val ues are, r espect i vel y, l ar ger or smal l er . I n
ot he r wor ds, i t makes no di f f er ence in s ummi ng t he ~ val ues whi ch
of a gr oup of i ndi vi dual s havi ng a cons t ant ~9~ s cor e ar e chosen f or
r ej ect i on or f or as s i gnment el s ewher e and, i n addi t i on, all t hos e ha v-
i ng such a cons t ant scor e ar e t o be pr e f e r r e d t o t hos e havi ng a l ower
score. Hence, wi t hi n any gi ven "sl i ce" i t makes no di f f er ence in max-
i mi zi ng F~.~i whi ch i ndi vi dual s a r e sel ect ed f or ot her as s i gnment s ,
al t hough t he r el at i ve pr opor t i ons chosen f r om successi ve sl i ces do
ve r y defi ni t el y af f ect ~' ~)~. Among t hos e not as s i gned el sewher e, a
l ower val ue woul d never be sel ect ed in pr e f e r e nc e t o a hi gher ~) val ue,
no ma t t e r wh a t scor es ar e obt ai ned on cr i t er i a of ot her as s i gnment s .
To s t at e ot her wi s e woul d be t o i mpl y t h a t 9~ i s not t he bes t pr edi ct i on
of y~. I n ot he r wor ds , t he f a c t t ha t we a r e concer ni ng our s el ves wi t h
~)~ i ns t ead of y l , t he cr i t er i on i t sel f, does not i nfl uence t he val i di t y of
t he f or e goi ng s t a t e me nt so l ong as t he ~ val ues r e pr e s e nt t he " bes t "
pr edi ct i on Obt ai nabl e f r om t he t e s t scor es i n t e r ms of whi ch t he in-
di vi dual s ar e pl ot t ed. Fr o m our a s s umpt i on of l i near r egr es s i on l i nes
i t ma y be st at ed, in a ny event , t ha t t he me a n cr i t er i on s cor e is equal
t o t he mean pr edi ct ed cr i t er i on s cor e f or a ny s egment of our "sl i ce. "
Fr o m t hi s as s umpt i on i t al so f ol l ows t ha t none of t he "sl i ces" is cur vi -
l i near. As t hes e "sl i ces" become i nf i ni t esi mal l y t hi n t he y a r e defi ned
exact l y b y a cr i t i cal scor e on 9~, si nce t he y consi st of i ndi vi dual s h a w
i ng t he s ame cons t ant .~ score. Not e t ha t no s t a t e me nt concer ni ng t he
ma gni t ude of t he cr i t i cal scor e has been made. I t i s mer el y s hown
t h a t t he boundi ng s ur f ace is defi ned by some cr i t i cal s cor e on ~)~. Not e
al so t ha t t he boundi ng s ur f ace is perpen~l i cul ar t o t he mul t i pl e re-
gr es s i on l i ne det er mi ni ng t he gi ven 9~ val ues. Thus we have dem-
ons t r a t e d t he f i r st of our t hr e e pr oposi t i ons, namel y, t ha t t he desi r ed
boundi ng s ur f aces di f f er ent i at i ng bet ween an as s i gned gr oup i and
t he r ej ect ed gr oup is defi ned by a gi ven cr i t i cal score. I t woul d pos-
si bl y be mor e a ppr opr i a t e t o s ay t ha t we have s hown t ha t i t f ol l ows
di r ect l y f r o m our as s umpt i on of l i near i t y of r egr es s i on l i nes and have
el abor at ed s ome wha t i t s meani ng and i mpl i cat i ons.
The demons t r at i on of t he second pr opos i t i on f ol l ows t ha t of t he
f i r st al mos t exact l y. I f we we r e t o pl ot i ndi vi dual s ha vi ng t h e s ame
di f f er ence scor es ( ~ - - Yj) we woul d obt ai n a ser i es of "sl i ces" or
HUBERT E. BROGDEN 145
bounding surfaces analogous to those obtained by "slicing" on 9~.
From our assumptions of linearity of regression lines these bounding
surfaces would also be linear. The necessity t hat each bounding sur-
face differentiating between any t wo assignments must be defined by
a constant value of difference variables (~j -- 9~) is evident when it
is realized that the differences between the ~ values are a direct indi-
cation of the gain or loss in I to be effected by shifting individuals
from one assignment to another. Thus the second of the three points
to be proved also follows almost directly from the assumptions of
linearity of regression lines.
The next and third point to be demonstrated is t hat I is max-
imized when each critical difference score is the difference between
the critical rejection scores on the ~ values of the two assignments
involved, assuming that the proportion of individuals in each assign-
ment remains constant. Suppose we consider the plane formed by
plotting paired ~ and ~j values. To obtain agreement with our first
proposition, assigned groups i and ] must be separated from those
rejected by critical scores on ~i and Yi We will refer to these criti-
cal scores as c~i and c9~. The rejection boundaries defined by the
critical scores would be straight lines, each perpendicular to its corre-
sponding axis. To obtain agreement with our second proposition the
t wo assigned groups must be separated from each other by one of a
family of lines defined by constant values of the difference ( ~ -- 9s).
Parenthetically the reader is reminded t hat the slope of all members
of this family of lines is the ratio between the s.d.'s of ~ and Ys, while
the s.d.'s are in t urn equal to the multiple correlation of the several
predictors for the given assignment times the s,d. of the criterion of
t hat assigmnent.
Suppose t hat the third proposition--which remains to be demon-
s t r at ed- has been assumed valid in separating the three groups and the
boundary between the t wo assigned groups is t hat one of the family of
difference lines identified by the difference between the t wo critical re-
jection scores or ,9~ -- egj. I f this were the case, the three boundaries
would have a common point of intersection. Since the critical differ-
ence score may be increased or decreased only, and since the critical
rejection scores are determined aft er each such change in the critical
difference score by the requirement t hat the number in each category
remains constant, it is clear that only these two t ypes of changes are
possible. Consider Fig~lre 1. Here, we have assumed t hat on the
plane formed by ~ and .~j the critical rejection score for ~ is 2.0;
the critical rejection score ~s 1.0 for ~)j, and the critical difference
score is 1.0. These are indicated by the heavy lines. Suppose t hat the
critical difference score were increased so as to t ransfer m individuals
146
2.0
A S S I S T i
. . . . . . . . . . I I I l l l l l f l l l l l l l l l l / / / / l l / / / / / ~ / / ~ l f l
P S YC HOME T R I KA
c 7 ~ 1. o
y j
F:GURZ 1
f r o m a s s i gnme nt i t o as s i gnment ] and changes we r e made Jn t he
cr i t i cal r ej ect i on scor es so as t o al l ow no change bet ween t he numbe r
i n a s s i gnme nt i and ] bef or e t he t r a n s f e r and t he n u mb e r obt ai ned
a f t e r wa r d s . The t hr e e shaded ar eas be t we e n t he pai r s of heavy and
dot t ed l i nes del i neat e t he i ndi vi dual s whos e a s s i gnme nt ~s changed.
I t is obvi ousl y t he r es ul t i ng changes i n t he cr i t er i on of t hei r assi gn-
me nt and t he consequent change i n t he cr i t er i on scor es of t hes e in-
di vi dual s t ha t wi l l change I . I n ~he s haded a r e a bet ween t he t wo
cr i t i cal di f f er ence scor es t he m i ndi vi dual s i nvol ved woul d have been
s hi f t ed f r om a s s i gnme nt i t o as s i gnment ] . Si nce, i n t he case of
t hose i ndi vi dual s di r ect l y on t he or i gi nal r ej ect i on lines ( t he heavy
l i nes i n Fi gur e 1) , t he di f f er ence be t we e n t i l e ~ and ~)j scor es was 1.0,
t he s hi f t of each such l ndi vi dual t o assi~o~ment i woul d decr ease t he
over - al l s um by exact l y 1.0, si nce t he cr i t er i on scor e f or such an in-
di vi dual on cr i t er i on i , t hei r ~s s i gnment a f t e r t he shi f t , is exact l y
1.0 l ar ger t han t hei r scor e on cr i t er i on ], or t ha t f or t hei r pr evi ous as-
si gnment . However , t he aver age di f f er ence in cr i t er i on scor es of t he
i ndi vi dual s s hi f t ed is s ome wha t l ar ger t han 1.0, si nce t he s hi f t in-
cr eas ed t he di f f er ence cr i t i cal score. He nc e t he decr eas e ef f ect ed in each
i ndi vi dual ' s cr i t er i on scor e woul d be 1.0 pl us an i ncr ement whi ch we
HUBERT E. BROGDEN 147
shal l call r i d . The ef f ect of t l d s s hi f t on t he over-aU s um woul d be
t o add, al gebr ai cal l y, - - ~ (1. 0 + d d ) . The i ndi vi dual s i n t he s haded
a r e a bet ween t he heavy and dot t ed l i ne i ndi cat i ng t he t wo cr i t i cal r e-
j ect i on scor es on Ys a r e bei ng s hi f t ed t o t he r ej ect ed gr oups. Si nce,
on t he aver age, t hei r Ys scor es a r e s ome wha t above 1.0, t hi s s hi f t adds
- - m( 1 . 0 + A9s ) t o I . By anal ogous r easoni ng, t he i ncl usi on of t hos e
i n t he t hi r d s haded ar ea adds m( 2.0 - - A~)~) t o I . The ne t change
is t hen - - m (1.0 + 3 d ) - - m (1. 0 +A9 s) + m( 2. 0 - - Ag,,), whi ch r educes
t o m( - - Ad - - 3~3~ - - A~)s), si nce m wa s ma de cons t ant i n or der t ha t t he
t ot al i n each as s i gnment woul d r e ma i n t he same.
Fi g u r e 2 i s t o be i nt e r pr e t e d i n t he s a me ma n n e r as Fi g u r e 1.
The s hadi ng i ndi cat es t he a r e a s af f ect ed b y t he changes i n t he cr i t i cal
di f f er ence scor es and c ompe ns a t i ng changes ~n t he cr i t i cal r ej ect i on
scores. I n t he s haded ar ea be t we e n t he t wo cr i t i cal di f f er ence scor es
t he mean di f f er ence is s ome wha t smal l er t ha n 1.0, so t ha t i n s hi f t i ng
i ndi vi dual s t o a s s i gnme nt i , t he change ef f ect ed i n I woul d be
m( 1. 0 - - r i d) . I n t he s ame wa y i t can be seen t ha t t he shaded a r e a
be t we e n t he he a vy and dot t ed l i nes f or 9~, whi ch r e pr e s e nt s i ndi -
vi dual s who have been s hi f t ed f r om as s i gnment ~ t o r ej ect i on, ha ve
an aver age scor e of s omet hi ng over 2.0, or s ay 2.0 + A~ . The l oss
occasi oned by s hi f t i ng t o t he r ej ect ed gr oup is - - m( 2 . 0 + A~ ) . I n -
~1 = 2. 0
/ /
~ I ~ T L
Fmt~RE 2
148 PSYCHOMETRIKA
di vi dual s i n t he cor r es pondi ng s haded a r e a f or ~ ; have an aver age cri -
t er i on scor e of 1.0 - - d~)j so t ha t t he ef f ect of changi ng t hese i ndi vi d-
ual s f r om t he r ej ect ed gr oup t o a s s i gnme nt ] is t o add m (1.0 - - Agj ) t o
I . The t ot al of t hes e sever al changes is agai n m ( - - Ad- Ay~ - - Ay 2) .
The a r e a i n whi ch i ndi vi dual s f or me r l y r ej ect ed a r e now gi ven assi gn-
me n t ] i n Fi gur e 1 and t he a r e a i n Fi gur e 2 whe r e i ndi vi dual s f or -
me r l y r ej ect ed a r e gi ven a s s i gnme nt i have been negl ect ed, si nce t he y
a r e s econd- or der di f f er ent i al s. I t is t r ue, of cour se, t ha t t he changes
i n t he cr i t i cal scor es of 9~ and 9J r equi r ed i n or de r t o hol d t he pr o-
por t i on of cases i n each a s s i gnme nt cons t ant wi l l me a n t ha t in ot her
pl anes t he cr i t i cal di f f er ence scor e f or , say, 9~ and ?)k or Yi and ?)kis no
l onger equal t o t he di f f er ence be t we e n t he cr i t i cal scor es on t he t wo
c r i t e r i a defi ni ng t he pa r t i c ul a r di mensi on i n quest i on and, i t mi ght
be ar gued, t he changes i n t he cr i t i cal scor es of ot he r var i abl es t ha t
wi l l be r equi r ed i n or der t o hol d t he numbe r i n each as s i gnment con-
s t a nt mi ght r ai s e I r a t h e r t ha n l ower it. Howe ve r , such changes a r e
exact l y anal ogous t o t hos e we have di scussed, and t he f or egoi ng p r o o f
wi l l appl y in t he case of t hes e f u r t h e r changes.
Si nce changes in cr i t i cal scor es f r om our pr opos ed maxi mal po-
si t i on can be of t wo t ypes only, and si nce i t has been s hown t ha t wi t h
each of t hes e changes a decr ease in I is effect ed, our t hi r d pr opos i t i on
has been demons t r at ed. When all cr i t i cal di f f er ence scor es a r e equal
t o t he di f f er ences bet ween t he t wo cr i t i cal r ej ect i on scores, no change
i n cr i t i cal di f f er ence scor es whi ch mai nt ai ns t he s ame pr opor t i on i n
each as s i gnment will ef f ect an i mpr ove me nt i n I .
Demons t r at i on of t hes e t hr ee pr opos i t i ons means t ha t wi t h se-
l ect i on of any set of cr i t i cal r ej ect i on scores, det er mi nat i on of cr i t i -
cal di f f er ence scor es f r om t he r ej ect i on scor es and as s i gnment wi l l
be such t ha t I is maxi mi zed f or t he pr opor t i ons of cases i n each of
t he assi gnment s. The sel ect i on of t he mos t f eas i bl e pr ocedur es of
det er mi ni ng cr i t i cal scor es such t ha t t he pr ope r numbe r of i ndi vi dual s
ar e al l ocat ed t o each as s i gnment is t he pr i nci pal r emai ni ng pr obl em.
I t has al r eady been i ndi cat ed t ha t a t heor et i cal sol ut i on t o his pr ob-
l em whi ch woul d have gener al appl i cat i on does not a ppe a r possi bl e,
si nce as s umpt i ons concer ni ng t he na t ur e of f r e que nc y sol i ds f or me d
by pl ot t i ng i ndi vi dual s on q- axes woul d have t o cor r es pond r a t h e r
cl osel y t o t he empi r i cal da t a i n or de r t ha t such a sol ut i on woul d be
usef ul . I n any event t he sol ut i on woul d be exceedi ngl y compl ex and
ve r y pr obabl y r a t he r c umbe r s ome f r om t he comput at i onal vi ewpoi nt .
The al t er nat i ve woul d a ppe a r t o be a successi ve appr oxi mat i on pr o-
cedur e
A defi ni t e s t ep- by- s t ep successi ve appr oxi mat i on pr oc e dur e wi l l
be pr esent ed. Al t hough t hi s pr ocedur e is not neces s ar i l y t he mos t
HUB E R T E . B R OGDE N 149
rapid, it is simple and easy to apply. Let us assume t hat we are start-
ing with a list of ?~ values for all assignments and all individuals. The
actual steps of the proposed procedure are as follows:
1. Prepare frequency distributions of ~ values for each assign-
ment, recording the identification number of each individual
instead of a frequency tally.
2. Draw a line through the top part of the frequency curve iso-
lating, in the segmented portion, the required number of
cases for t hat assignment. The point of cut of this line is
the fi rst estimate of the critical rejection score (designated
c 9 , ) .
3. Compute critical difference scores (c9~ - - ~j ) .
4. Identify in each frequency distribution, by referri ng the
identification number of selected individuals to the listing of
values and to the critical rejection scores, all individuals
above ~9 in other frequency distributions.
5. At the time t hat the ~) values are located in the listing, r ef er
to the critical difference scores and determine in which as-
signment the individual belongs. Indicate in some way on the
individual' s identification number in the frequency distribu-
tion t hat he is or is not to be included in t hat assignment.
6. Lower the critical scores to include enough additional sub-
jects to replace those assigned elsewhere.
7. Determine whet her any of the new assignees were included
(either in the first or second selection) in any other assign-
ment.
T AB L E 1
L i s t i n g o f y Va l u e s i n Nu me r i c a l E x a mp l e
Ma n y,~ Va l u e s Ma n y ~ Va l u e s
No. ~ ~ 7. L No. ~ ~ ~-.. ~ ,
1 - - . 8 - - 2 . 4 . 1 - - . 4 1 1 - - . 7 - - 1 . 2 . 3 . 1
2 h . 3 - - 1 . 9 . 3 . 7 1 2 - - . 9 - - 1 . 1 - - . 7 - - 1 . 4
3 - - . 4 - - 1 . 8 . 1 . 3 1 3 - - . 3 - - 1 . 1 . I - - . 9
4 - - 1 . 1 - - 1 . 7 . 6 1 . 1 1 4 - - . 8 - - 1 . 0 - - . 3 . 5
5 - - . 1 - - 1 . 6 - - . 8 - - 1 . 0 1 5 - - . 4 - - 1 . 0 . 6 0 . 0
6 - - . 5 - - 1 . 5 . 7 - - . 3 1 6 - - . 3 - - 1 . 0 - - . 5 . 3
7 - - . 7 - - 1 . 4 - - , I - - 1 . 3 1 7 . 1 - - . 9 - - - : 1. 0 2 . 4
8 - - . 6 - - 1 . 3 . 2 . 4 1 8 - - . 5 - - . 9 . 8 . 9
9 - - . 1 - - 1 . 3 - - . 6 - - . 4 1 9 . 2 - - . 9 . 5 - - . 5
1 0 - - . 2 - - 1 . 2 - - . 9 - - 1 . 1 2 0 - - . 2 ~ . 8 . 2 1 . 2
150 P S YC HOME T R I KA
T a b l e 1 ( C o n t i n u e d )
L i s t i n g o f y Va l u e s i n Nu me r i c a l E x a mp l e
Ma n Yi Va l u e s Ma n ~4i Va l u e s
No . Yl Y2 ~J Y4 No . y , Y2 Y~
2 1 - - . 7 - - . $ . 6 . 9 6 1 - - . 5 . 3 . 4
. e - - . 8 - - . 2 . 9 6 2 - - . 1 3 . 3
2 3 . 3 - - . 7 . 2 - - . 1 6 3 - - . 3 . 3 - - . 4
2 4 . 7 - - . 7 - - . 6 . 2 6 4 . 1 . 3 . 2
2 5 - - . 2 - - . 7 . ~ . 6 6 5 4).0 . 4 - - . 4
Y4
1 . 0
0 . 0
- - 1 . 8
- - . 9
- - . 2
2 6
2 7
2 8
2 9
3 0
, 2
- - . 9
- - . 1
- - , 6
. 5
- - . 6
. 6
- - . 6
- - . 5
, 5
. 2 . 8 6 6 . 6 . 4 . 1
. 1 - - . 3 6 7 1 . 0 . 4 0 . 0
- - . 2 - - . 5 6 8 . 4 . 4 0 . 0
- - 1 . 1 - - 1 . 7 6 9 . 6 . 5 - - . 7
- - . 5 - - . 9 7 0 . 4 . 5 1. 1
. 4
1 . 8
0 . 0
. 8
1 . 3
3 1
3 2
~33
3 4
3 5
. 2
- - . 2
- - . 3
. 1
- - . 5
- - . 5
- - , 4
- - , 4
o 4
- - . 1 . 2 7 1 - - . 2 . 5 . 1 - - . 8
0 . 0 1 . 0 7 2 . 9 . 5 0 . 0 - - . 6
3 - - . 7 7 3 - - . 1 . 6 0 . 0 1 . 2
. 4 1 . 5 7 4 0 . 0 . 6 . 9 1 . 9
. 5 . 3 7 5 . 1 . 6 . 4 . 4
3 6 . 4 - - . 4 . 7
3 7 - - . 5 - - . 3 - - . 1
3 8 - - . 4 - - . 3 0 . 0
3 9 . 7 - - . 3 - - . 1
4 0 . 1 - - . 3 - - . 3
4 1 - - . 2 - - . 2 . 4
4 2 - - . 3 - - . 2 . 3
4 3 - - 1 . 0 - - . 2 . 2
4 4 - - . 1 - - . 2 . 7
4 5 - - . 1 - - . 1 - - . 3
4 6 . 2 - - . 1 - - . 9
4 7 - - . 1 - - . 1 . 5
4 8 - - . 4 - - . 1 1 . 0
4 9 . 2 0 . 0 - - . 2
5 0 . 3 0 . 0 . 5
5 1 . 6 0 . 0 - - . 1
5 2 0 . 0 0 . 0 . 2
5 3 . 3 . 1 . 4
5 4 . 5 . i . I
5 5 . 7 . 1 0 . 0
5 6 . 3 . 1 . 4
5 7 - - . 3 . 2 - - . 3
5 8 - - . 6 . 2 - - . 7
5 9 - - . 1 . 2 - - . 8
6 0 . 4 , 2 - - . 5
. 1 7 6 . 8 . 7 . 6 - - 2 , 4
- - . 6 7 7 . 1 . 7 - - . 4 - - . 1
. 4 7 8 0 . 0 . 7 . 1 . 5
. 6 7 9 . 1 . 8 . 3 1. 0
- - 1 . 0 8 0 0. O . 8 - - . 5 - - . 3
1 . 7 8 1 . 1 . 8 - - . 1 0 . 0
. 8 8 2 . 8 . 9 0 . 0 - - . 1
- - . 2 8 3 - - . 2 . 9 - - . 2 . 2
- - . 7 8 4 . 3 . 9 - - . 6 . 5
- - . 2 8 5 0 . 0 1 . 0 - - . 6 - - 1 . 3
- - . 6 8 6 . 2 1 . 0 - - . 4 . 5
- - 1 . 0 8 7 . 5 1 . 0 - - . 1 - - . 2
- - . 5 8 8 . 4 1 . 1 - - . 2 - - 1 . 2
- - . 7 8 9 . 3 1 . 1 - - . 3 - - . 4
1 . 6 9 0 0 . 0 1 . 2 . 8 1. 3
- - 1 . 6 9 1 . 9 1 . 2 0 . 0 1 . 1
- - . 5 9 2 0 . 0 1 . 3 - - . 2 - - 1 . 5
- - . 4 9 3 1. 1 1 . 3 - - . 1 - - . 1
. 8 9 4 . 1 1 . 4 - - . 3 - - 1 . 9
. 6 9 5 - - . 4 1 . 5 - - . 3 . 3
. 7 9 6 . 2 1 . 6 - - . 2 - - . 8
- - - 1 . 1 9 7 0 . 0 1 . 7 - - .4~ . 2
- - . 3 9 8 . 3 1 . 8 . 1 1 . 4
7 9 9 4).0 1 . 9 . 1 . 1
- - 1 . 2 1 0 0 . 5 2 . 4 0 . 0 . 1
HUBERT E. BROGDEN 151
8. I f so, r epeat s t eps 5, 6, and 7 unt i l t he r equi r ed numbe r a r e
i n each a s s i gnme nt and no i ndi vi dual i s in mor e t han one
assi g]l ment .
A numer i cal exampl e will s er ve t o i l l ust r at e t hi s pr ocedur e. Le t
us as s ume t hat 13, 15, 5, and 15 per cent of t he popul at i on of appl i -
cant s ar e desi r ed f or , r espect i vel y, as s i gnment s 1, 2, 3, and 4. The
numer i cal val ues of t he ~) val ues a r e gi ven in Tabl e 1 and t he f r e-
quency di s t r i but i ons t o be pr epar ed as st ep 1 a r e gi ven in Tabl e 2.
The heavy lines on t he di s t r i but i ons i ndi cat e t he f i r st sel ect i on. The
cr i t i cal scor es are, r espect i vel y, .6, 1.0, .8, and 1.0. Thi s compl et es
st ep 2. The cr i t i cal di f f er ence scor es obt ai ned as st ep 3 ar e:
As s i gnme nt i As s i g n me n t ]
2 3 4
1 .4 .2 .0
2 - - . 2 - - . 4
3 --. 2
Not e t ha t t he or der of t he t we var i abl es in s ubt r a c t i ng det er mi nes
t he si gn of t he di fference. As s i gnme nt is made t o t he fi rst of t he t wo
as s i gnment s when t he val ue of t he di f f er ence var i abl e exceeds t ha t
of t he di f f er ence score.
I dent i f i cat i on numbe r s a ppe a r i ng in mor e t han one f r equency
di s t r i but i on have been pr i med. A line has been r ul ed t hr ough num-
ber s when assi gmment is el sewher e ( Tabl e 2) . Thi s compl et es s t ep 5.
Beyond t he f i r st appr oxi mat i on and r eas s i gnment , l i t t l e addi t i on-
al l abor is i nvol ved in at l east t hi s numer i cal exampl e. In a s s i gnme nt
1, man numbe r 67 ( t he onl y i ndi vi dual r eas s i gned) was r epl aced by
man number 30. I n a s s i gnme nt 2, men numbe r s 90, 91, and 93 we r e
r eas s i gned and r epl aced b y men numbe r s 83, 84, and 85. I n assi gn-
me nt 3, man numbe r 74 wa s r eas s i gned and r epl aced b y man numbe r
90. However , a f t e r r e a ppr oxi ma t / ng t he cr i t i cal di f f er ence s cor e be-
t ween as s i gnment s 3 and 4, ma n numbe r 90 was assi gned t o 4 ( or
r emai ned t her e) . Hence ma n numbe r 6 wa s t he event ual r epl ace-
ment . In as s i gnment 4 men number s 70, 91, and 98 wer e r eas s i gned
and r epl aced by men numbe r s 18, 21, and 79. However , 18 and 21
wer e assi gned t o, 3 and 1, r espect i vel y, and a f t e r t he cri t i cal di ffer-
ence scor es wer e r e a ppr oxi ma t e d t hey still r emai ned t her e. Hence
man number 26 was pl aced in as s i gnment 4.
I f i t .is desi r ed t o speed t he pr ocedur e by ma ki ng s ubj ect i ve al-
l owances f or addi t i onal f act or s i n a r r i vi ng a t appr oxi mat i ons t o t he
cr i t i cal scores, t he f ol l owi ng gener al pr i nci pl es ma y be hel pf ul :
1. As s i gnment s whos e 9 val ues have r el at i vel y l ar ge s t a nda r d
152 PSYCHO?,{ETRIK A
devi at i ons will t end t o have hi gh cr i t i cal scores, si nce t he
r el at i ve pr opor t i on of t he i ndi vi dual s i n t he over l appi ng
ar eas, i.e., ar eas above t he cr i t i cal scor es of any pai r of
val ues, assi gned t o a gi ven catego~':," wi l l depend upon t he
r el at i ve size of t he s. d. ' s of t he ~ val ues.
2. Si nce degr ee o f over l appi ng is a f unct i on of degr ee of i nt er -
cor r el at i on, all cr i t i cal scor es will inc-' ease as t he degr ee of
i nt er cor r el at i on of ~3 val ues . decr eas es .
3. I n maki ng al ) pr oxi mat i ons beyond t he first, i t shoul d be re-
member ed t ha t when s hi f t s ar e made in cr i t i cal r ej ect i on
s cor es so as t o i ncr ease t he numbe r i n a gi ven cat egor y, t he
changes in t he cr i t i cal differ(~nce scor es whi ch aut onmt i cal l y
fol l ow will decr eas e t he numbe r in t he r e ma i ni ng assi gnment .
I t is qui t e possi bl e t hat t he a ppr oxi ma t i on pr ocedur es woul d be
much mor e l abor i ous i f cr i t i cal scor es we r e l owe r a n d / o r t he size
of t he sampl e l ar ger t han in the. numer i cal exampl e. The a ut hor does
not feel t hat i t .is f easi bl e t o of f er defi ni t e i nform~. t i on concer ni ng t he
l abor r equi r ed i n t he var i ous t ype s of s i t uat i ons i n whi ch t he pr oce-
dur e s mi ght be appl i ed. T h e pr obl em i l l us t r at ed was, however ,
changed t o t he ext ent of r equi r i ng 25% in each a s s i gnme nt and t he
sol ut i on obt ai ned by t he a ut hor in less t ha n one hour ' s t i me.
i

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

I

~
D
l
~
)
-
~
O
~
O
a
b
3
1
~
O

'
'
~
D

"

"

'

"

"

"

I

@

=

~
1
"
~

~

~
l
~
'

~

'.
.
0
~

~

~

0
0
I

I
D

~

t
o

.
~

.
.
.
.
.
.

~
r
r
y
~
r
r
~

~
'

~
~
~

~

~

0
~
-
~
-
~
-
~

~
I
~

~

o

o

~

~

~

~

o

o

I

c
o

t
~

o

o

t

~

o

i
c
~

0

,
~
-
~
o

~

~
o
~

o
~
o
~

~
t
o
-
~

~

~

o
~
t
o
,
~

K

~

c
o

o

"

o
~
0
~
o
~
o
0
~
o
~

~
0
~
~
0
~

~
D

i
-
~
O
D

l
.
a

0

o

~

O
~

~
.
n
~
I
~
O
~

-
q

~

~

I
0

I
x
~
~

-
,
I

O
~

O
~

O
~

I
-
~

O
~

~

~
I
0
~

q
~

~

"
,
0

~

~

o
o

i
'
-
'

,
~

m
m

'.
~

~
o
~

"
-
a

~

o
~

i
-
'

o
~

~

"
~

~
l
.
~
.
.
l
~

,

d
~
.

0
~
,

I
D

i,
~
.

-
,
1

i1
~
.
(
.
r
l

0
1

0
:
)

(
3
~

O
~

0

0
:
:
,

f
.
.
~

O
~

O
~

0
1

0
1

0
1
1
'
0

"
~
1

..,,

o

0