7/28/2019 gescheider 56-64

1/9

56 3. PSYCHOPHYSICAL THEORY

relation between the empirically determined proportion of hits and false alarmsshould be linear. The straight lines start from various values of p(yeslstimulus)when p(yeslno stimulus) is zero. The higher the signal strength, the higher thevalue of p(yesl stimulus) will be. As the observer increases his guessing rate,p(yeslstimulus) andp(yeslno stimulus) should increase linearly until both become1.0. A family ofROe curves predicted from threshold theory is seen in Figure 3.8.It is important to note that if Equation (3.5) is applied to the values of p(yeslstimulus) on the ordinate, all of the curves become horizontallines illustrating theassumed independence of p*(hits) and p(false alarms). It is unfortunate for theproponents of threshold theory that ROe curves have been found to deviateconsistently from linearity. The curved shape of empirically obtained Roe curvesis thought to constitute powerful evidence for rejection of the threshold concept infavor of a new conception of the observer's behavior in the detection situation(Swets, 1961). We must discard the theory of a threshold that is exceeded onlywhen a stimulus of sufficient strength is presented. But the concept of a lowerthreshold that is frequently exceeded by spontaneous activity in the nervous systemis not in disagreement with the shape of empirical ROe curves. Later in the chapterlow threshold theory will be discussed.

THE THEORY OF SIGNAL DETECTIONThe discovery that expectancy and payoff have such a dramatic influence upondetection behavior has been incorporated into a new theoretical conception of thedetection situation. Tanner and Swets (1954) proposed that statistical decisiontheory and certain ideas about electronic signal-detecting devices might be used tobuild a model closely approximating how people actually behave in detectionsituations. The model is called the theory of signal detection (TSD) and isdescribed in detail by Green and Swets (1966).

Signals (stimuli) are always detected-whether by electronic devices or byhumans-against a background level of activity. The level of this backgroundactivity, called noise, is assumed to vary randomly and may be either external tothe detecting device or caused by the device itself (e.g., physiological noise causedby spontaneous activity of the nervous system). In the detection situation theobserver must therefore first make an observation (x) and then make a decisionabout the observation. On each tria! the observer must decide whether x is due to asignal added to the noise background or to the noise alone. When a weak signal isapplied, the decision becomes difficult and errors are frequent. One factor con-tributing to the difficulty of the problem is the random variation of backgroundnoise. On some trials the noise level may be so high as to be mistaken for a signaland on other trials it may be so low that the addition of a weak signal is mistaken fornoise. This state of affairs can be represented graphically by two probabilitydistributions describing the random variation of noise (N) and the signal plus noise(SN) (Figure 3.9). Since the signal is added to the noise, the average sensory

7/28/2019 gescheider 56-64

2/9

fN (X ) fSN(X)

c-,< IeID"O

. : = - fN (X ) fSN(X)--.oa.ooc t

fN {X ) fSN(X)

THE THEORY OF SIGNAL DETECTION 57

Magnitude of sensory observation (x )FIGURE 3_9 Theoretical frequency distributions of noisefN(x) , and signal plus noise fs~(x) for threedifferent values of signal strength.

observation magnitude will always be greater for the signal-plus-noise distribu-tion,lsN(x), than for the noise distribution.fwa )_However, the difference betweenthe means becomes smaller and smaller as the signal strength is decreased, until thedistributions are essentially the same. It is when the two distributions greatlyoverlap that decision-making becomes difficult.

On a specific trial the observer makes a sensory observation x which consistsof a sample from one or the other of the distributions and is required to decide onthe correct distribution. The ordinate of !N(X) gives the probability density,' orlikelihood, of x occurring when only noise is presented. Similarly, the ordinate of!SN(X) gives the likelihood of x occurring when a signal is presented. Each value ofx can now be expressed in terms of these two likelihoods or probability densities.For each value of X there exists a particular likelihood ratio, l(x), defined as

l(x) = !SN(X)!N(X) (3.6)

The Iikelihood ratio provides the observer with a basis for making a decision sinceit expresses the likelihood of x in the SN situation relative to the likelihood of x in

'The term probability density is used because x is continuous rather than discreteo If x had a limitednumber of discrete values each could be described as having a particular probability of occurrence. Ineither case the ordinate gives the relative likelihood of a particular value of X.

7/28/2019 gescheider 56-64

3/9

1 1 No " reg i on "Yes" region

58 3. PSYCHOPHYSICAL THEORYCriterion

Magnitude af sensory observation (x )FIGURE 3.10 Theoretical frequency distributions of noise,fN(x), and signaJ plus noise,fsN(x). Thelocation of the observer's criterion determines whether a particular sensory observation results in a"no" or a "yes " response.

the N situation. Even though X may vary on several dimensions (e.g., hue,saturation, brightness, shape), each X can be located on a single dimension oflikelihood ratio since for each X there exists single values of f N ( X )and f s (x). Thus,the observer' s final decision of whether x is due to N or SN can be based on a singlequantity.

One of the assumptions of TSD is that an observer establishes a particular valueof l(x) as a cutoff point, or criterion (/3), and that his decision will be determinedby whether or not a particular observationx is above or below the criterion. Propo-nents of the theory assume that the observer operates by a decision rule: when l(x)is equal to or greater than f 3 he should choose SN, and when [(x) is below f 3 heshould choose N (Figure 3. 10). If the observer properIy sets his criterion he wilIperform optimally in a long series of observations.

Swets, Tanner, and Birdsall (1961) consider the detection situation to beanalogous to a game of chance in which three dice are thrown. Two of the dice areordinary, but the third is a special die with three spots on each of three sides and nospots on the other three sides. When the dice are thrown, the player is told only thetotal number of spots on alI three dice. This informaton is analogous to theinformation given for each observation in a detection situation. On the basis of thetotal number of spots showing, the player must decide whether the unusual dieshowed a zero or a three. Similarly, in the detection situation the observer mustdecide whether his observation was a product of noise alone or of signal plus noise.To come out ahead in the long run the player of the dice game would compute theprobability of occurrence of each of the possible totals (2 to 12) when the unusualdie shows zero and, likewise, the probabilities of each of the totals (5 to 15) whenthe unusual die shows three. The results could be plotted as two probabilitydistributions and should be thought of as the analogs of the noise and signal-plus-noise distributions (Figure 3.11). Furthermore, as in the detection situation a

7/28/2019 gescheider 56-64

4/9

THE THEORY OF SIGNAL DETECTION 59criterio n should be set so that if the total number of spots were greater than somenumber the player would say "three," and if the total were les s than the number hewould say "zero. " In our example, where the probabilities of a three and a zero areboth .50 and the costs and values are the same for the various decision outcomes,the optimal criterion is the point where the two curves cross. In a detectionsituation, where the stimulus probability is .S O and the costs and values are equalfor the various decision outcomes, the optimal criterion is also the point on theobservation magnitude dimension where the two distributions cross,

It can be demonstrated mathematically that in the dice game the optimal cutoffpoint changes when the conditions of the game are changed. For example, if theunusual die were changed to one having three spots on five of the six sides and nospots on only one side, the probability of obtaining three spots would be .83 insteadof .50 and the optimal criterion would be lowered. A good player would lower hiscutoff point for saying "three." Likewise the optimal criterion would be raised ifonly one side of the die had three spots and the other five sides had none. In thedetection situation, changing the stimulus probability is assumed to have a similareffect on the observer' s criterion. In order to perform optimally in a situation wherestimulus occurrence is highly probable, an observer will report a signal after a lessintense sensory observation than when the stimulus is relatively improbable.

The Iocation of the optimaI cutoff point in the dice game is also influenced bychanges in the payoff conditions, For exampIe, if the reward is great for correctIysaying "three" and the punishment slight for saying "three " when "zero" iscorrect, the optimal criterion wilI, of course, be reIativeIy low for saying "three."Rewards and punishments in the detection situation are assumed to have a similareffect on the observer's criterion. A radar scope observer, for instance, maintains alow criterion for reporting signaIs because of the extreme importance of detectingenemy aircraft and the possibIe disastrous consequences of failing to do so.

Ql .20ueo. . .o .15uo

r-,1 IO r-.J L__

Ir-.JIr-.J

1r-.JI_.J

3. . .O .10>--.oo .05.oO. . .Q..

2 3 4 5 6 7 8 9 10 11 12 13 14 15Totol of three dice

FIGURE 3.11 Probability distributions forthe dice game. (From Swets, Tanner, &Birdsall, 1961.Copyright 1961 by the American Psychological Association. Reprinted by perrnission.)

7/28/2019 gescheider 56-64

5/9

60 3. PSYCHOPHYSICAL THEORY

The criterion set in the dice game will also be influenced by the degree of overlapof the two distributions. If the number of spots on the unusual die is made zero orfour instead of zero or three, the distributions will overlap less because thedistribution when four is correct will be shifted further up the scale, and the gamebecomes easier. Since the point where the two distributions cross is also shifted upthe scale, the optimal criterion will be higher. This manipulation in the dice gamecan be translated into an increase in stimulus intensity in the detection situation.When the signal-plus-noise distribution is shifted to a higher point on the obser-vation magnitude dimension, detection becomes easier and the optimal criterion ishigher.

In summary, the detection of energy changes in our environment involves,according to TSD, the establishing of a decision rule in the same way as theefficient playing of a game of chance does. The decision rule is the setting of acriterion determining which hypothesis about a given piece of information will beaccepted and which rejected. The location of the optimal criterion is a function of(a) the probabilities of the N and SN presentations, and (b) the costs and values forthe various decision outcomes.

In the detection situation, where the costs and values of the various decisionoutcomes and the probability of signal presentation are precisely known, theoptimum value of the criterion, { 3 o p , can be caIculated by

_ p(N) value(correct rejection) - cost(false alarm) (3.7){ 3 o P ! - p(SN) x value(hit) - cost(miss) .

{ 3 o P ! is the value of the likelihood ratio, l(x), which, when used as the criterion, wilIresult in the largest possible winnings in the long run; p(N) is the probability of anoise trial;p(SN) is the probability of a signal-plus-noise trial; value is the amountgiven to the observer for each correct decision; and cost is the amount taken awayfrom the observer for each incorrect observation.

When the value of {3, as calculated from the judgments of observers, is com-pared with { 3 o P ! , it is generalIy found that observers do fairly well at optimizingtheir winnings. An exception to this rule occurs, however, when { 3 o P ! is very smallor very large, in which case { 3 will not be as extreme as { 3 o P t and the observer wilIfail to optimize his winnings. Observers refuse to set extremely low or extremelyhigh criteria even though the conditions of the situation c1early demand such astrategy for optimal performance.

THEORETICAL SIGNIFICANCE OF THERECEIVER-OPERATING CHARACTERISTIC CURVE

One of the main sources of evidence supporting TSD is the experimental ma-nipulation of variables resulting in data plotted as Roe curves. In fact, shapes ofROe curves for various stimulus intensities can be generated from the postulates of

7/28/2019 gescheider 56-64

6/9

THEORETICAL SIGNIFICANCE OF THE ROC 61

1.0.9.8.7.6.5.4.3.2.1z Of)

(/)Ql>.a.

I d ' - I.0. - ~- - ~- - - ~-------r----~'---/----~

'-T----~

. 1 . 2 . 3 4 . 5 . 6 7 . 8 . 9 I D

.2 .3 .4 .5 .6 .7 .8 .91.0p yes lN)

FIGURE 3.12 The relation between Roe curves and lheoretical noise and signal plus noisedistributions. Variation in the observer 's criterion results in different points aJong the sarne ROe curvewhile variation of signal strength produces different ROe curves.

the theory and checked against empirical data. It should be recalled that the highthreshold theory was rejected because of its failure to predict empirical ROecurves.

The manner in which ROecurves are predicted from TSD is illustrated in Figure3.12. The upper ROe curve represents a situation where the signal strength issufficient to result in only a slight overlap of the N and SN probability distribu-tions. The vertical lines represent the locations of the criterion that might beassociated with specific conditions of stimulus probability and payoff. Each pointon an ROe curve, according to the theory, is determined by the location of theobserver's criterion on the x dimension. If an observation is to the right of thecriterion the observer will say "yes. " The proportion of the area under the curve to

7/28/2019 gescheider 56-64

7/9

62 3. PSYCHOPHYSICAL THEORYthe right of the criterion gives the proportion of "yes" decisions. Therefore, thevalues of p(yes/N ) and p(yes/SN ) can be determined by finding the areasunder the N and SN distribution curves, respectively, which are located to theright of the criterion. As the criterion is changed from high to low, the values ofp(yes/N ) and p(yes /SN ) change and, when plotted, form an ROe curve. Theillustration in the lower half of Figure 3.12 is a predicted ROe curve when thesignal strength is so weak: as to result in considerable overlap of the N and SN dis-tributions. The finding that the theoretical ROe curves generated in this mannerare very similar to those obtained experimentally is strong support for the theory.

In the classical psychophysical experiment the results expressed as thresholdswere a function of both stimulus detectability and the location of the observer'scriterion. Thus, as a measure of sensitivity to stimuli the threshold may behopelessly contaminated by changes in the observer's criterion. Such contamina-tion can lead to faulty conclusions about the results of an experiment. For example,there have been cases in which investigators incorrectly attributed large changes inthresholds to changes in the sensitivity of sensory pracesses. In fact, as revealed bysubsequent experimentation, the only thing that had changed was the criterion.TSD and its associated methodology afford a means of independently measuringeach of these factors. The theory proposes that d'; a measure of detectability, isequal to the difference between the means of the SN and N distributions ( M S N -MN) expressed in standard deviation units of the N distribution (O"N):

d' =M S N - M N .O"N (3.8)

Because the location of the SN distribution with respect to that of the N distributionis entirely a function of stimulus intensity and properties of the sensory system, d'is apure index of stimulus detectability which is independent of the location of theobserver's criterion.

But how can this theoretical concept of signal detectability be measured? Sincediffcrent d' values predict different ROe curves, the value of d' in a particularsituation can be ascertained by determining on which member of the farnily ofROe curves an observer's response probabilities fallo A farnily of Roe curvescorresponding to d' values ranging frorn Oto 3.0 is seen in Figure 3.13. Becauseonly a limited number of curves are generally presented in such a graph, it is best touse them when only approximate values of d' are needed. Fortunately, simplemethods are available for the determination of exact values of d'. The value of d'can be quickly derived from the empirical values of p(yes/S N ) and p(yes/N).By the use of a table of the normal distribution found in any textbook on statistics,these proportions are simply converted into z scores. To obtain d', the z score forfalse alarms is subtracted frorn the z score for hits. The value of d' can also bedetermined directly for any combination of false alarm and hit proportions by usinga table provided by Elliot (1964).

7/28/2019 gescheider 56-64

8/9

THEORETICAL SIGNIFICANCE OF THE ROC 63

z(f)

a.

.2. 1

.1 .3 4 .5 .6 .7 .8 .9 1.0P (yes IN)

FIGURE 3.13 A family of Roe curves corresponding 10 d' values ranging from o lo 3.0.By reference to a table of the normal distribution, the p(yesISN) and p(yesIN)

values of Figure 3.12 can be converted to Z scores. If this is done it wilI be seen thatthed' value for each pair of z scores wilI be 2.0 for the upper graph and 1.0 for thelower graph. Furthermore, it should be clear that for a particular separation of theN and SN distributions the value of d' will remain constant for alI possible criterionpositions. Thus, an ROe curve is a description of performance changes which areaccounted for by a constant d' and a continuously variable criterion.

It has been experimentally demonstrated for both visual stimuli (Swets, Tanner,&BirdsalI, 1955) and auditory stimuli (Tanner, Swets, &Green, 1956) thatd', asa measure of sensitivity, is not contaminated by the effects ofvariables which shiftan observer's response criterion. Furthermore, d' values, unlike the differentthreshold values obtained through the use of the various cIassical psychophysicalmethods, remain relatively invariant when measured by different experimentalprocedures. When observers were required to say "yes" or "no" in response to adesignated time interval that sometimes contained a signal, d' estimates werefound to approximate those obtained when the observer had to choose one of twodesignated intervals, one of which always contained a signal (Swets, 1959; Tanner&Swets, 1954).

Once the correct ROe curve has been determined, the location of the observer'scriterion, f3, can be determined by observing exactly where on the ROe curve thepoint is located. If the point is near the bottom of the Roe curve where the slope isgreat, the criterion is high; if the point is near the top of the curve where the slope is

7/28/2019 gescheider 56-64

9/9

64 3. PSYCHOPHYSICAL THEORYslight, the criterion is low. The exact value of f 3 is equal to the slope of the Roecurve at a particular point.

To reiterate, f 3 is a value of the likelihood ratio. It is the ratio of the ordinate ofthe SN distribution at the criterion to the ordinate of the N distribution at thecriterion, as follows:

f 3 !SN(X) at criterion!N(X) at criterion (3.9)

Figure 3.10 ilIustrates that moving the criterion to the right increases the value of f 3and moving it to the left decreases f3 . A low value of f3 represents a lax criterionwhere the observer will be liberal about reporting signals, while a high value of f 3represents a strict criterion where the observer will be conservative about reportingsignals.

The value of f3 can be caJculated from a pair of hit and false alarrn rates. Theordinate of the N distribution at criterion can be estimated as the ordinate valuegiven in the table ofthe normal distribution that corresponds to the false alarrn rate.Likewise the ordinate of the SN distribution at criterion is obtained by convertingthe hit rate into the ordinate value on the normal distribution curve. For example,ordinate values for a false alarm rate of .20 and a hit rate of .85 are .2801 and.2333, respectively:

.2333f 3 =.2801 =.83.If the investigator wishes to study the effects of a particular variable, he is

equipped with a technique for finding out whether the effects of the variable are ondetectability or on the location of the criterion. He has only to observe whethersystematic changes in the variable result in different points along a single ROecurve or points located on different Roe curves. A Iso the values of d ' and f3 can becaJculated for various experimental conditions. In some experiments manipulationof an independent variable has led to changes in both f 3 and d ' .

TESTING THE ASSUMPTIONS OFTHE THEORY OF SIGNAL DETECTION

The form of the ROe curve predicted fram TSD can be more easily subjected toexperimental tests if the values of p(yesISN) and p(yesIN) obtained in an ex-periment are plotted on the ROe curve as Z scores. If the N and SN distributions arenormal in form and also have equal variances (1'2), the ROe curves should belinear with a slope of 1.Owhen z scores for hits are plotted againstz scores for falsealarms. In the normal distribution and equal variance situation, when the criterionis shifted by a particular Z score distance on the N distribution, it is also shifted by