Here is an example of probit analysis so you can see how all these elements go together.
This experiment was done to find out what the threshold level of detection for image compression was. An image was compresses a five different levels. The compression, JPEG, is a lossy compression, meaning that information in the image is lost when the image is compressed. This may affect the appearance of the image. For a particular image, a method of constant stimulus experiment was performed to find out the level of compression that produces the just noticeable change in the image. The MCS was implemented with a forced choice procedure and it was analyzed using probit analysis on SAS.
Each compressed image is presented with the original to the subjects twice so that each subject has 10 trials. The stimulus pairs were constructed before the experiment. The placement of the images were random and the order of the stimuli were placed in a fixed random order. If you were doing this on a computer, you could have different random placements and orders for each subject
The following shows the subjects' instructions:
Instructions:
There are 10 sample pairs. In each pair of images, one is an original
and the other has been compressed to some degree using a JPEG algorithm.
Choose which image (A or B) is the original and record your response on
the attached response form.

The following shows the response form that the subject uses to record his responses and the scoring key which the experimenter has used for recording the subject's responses.
Response
Form


PAIR

CHOICE

PAIR

A

B

Correct


11

A

B

11

3

0

B


12

A

B

12

0

5

A


13

A

B

13

5

0

B


14

A

B

14

4

0

B


15

A

B

15

0

1

A


16

A

B

16

0

2

A


17

A

B

17

2

0

B


18

A

B

18

0

4

A


19

A

B

19

1

0

B


110

A

B

110

0

3

A

Image

QF

CR

Size(kb)

0 (original)

NA

1.0

432

1

1

3.0

142

2

14

9.0

48

3

29

13.9

31

4

44

17.9

24

5

114

31.9

13

The bottom table shows the identity of each image and the amount of compression. Image 0 is the original. The QF is a number that was taken from the software that does the compression. It assigns a JPEG Quality Factor to the image for different levels of compression. A lower QF corresponds to less compression. We will use this as a stimulus strength variable. We will also use the compression ratio (CR) as a value for the magnitude of the stimuli. The compression ratio shows the ratio of file size for the original compared to the compressed version.
So for stimulus 15 for example, the left image is the original and the right image is image one which has a QF = 1 and a CR = 3. If the subject could tell them apart, he would have picked B. If the subject could not see a difference he would have had a 50% chance of picking A or B. For each subject you go through and tabulate their results and then calculate the number of times all the subjects chose the compressed image for each sample pair. Sample pairs 14 and 18 contain the same images but are in different positions so the experimenter would keep track of which sample, A or B, was the compressed image for each pair.
Eighteen observers took part in the experiment. This means that each pair was presented 36 times (2x per subject x 18 subjects). The raw data looks like this:
JPEG Experiment Tally Sheet


Image

QF

CR

# Correct

# Observations

1

1

3.0

20

36

2

14

9.0

25

36

3

29

13.9

30

36

4

44

17.9

31

36

5

114

31.9

36

36

These are the data used in the probit analysis. There are two physical measures of stimulus magnitude (the independent variable), QF and CR. A probit analysis will be performed with each.
Here is one way to perform the probit analysis using SAS. First you need to create a text file. We will call the text file whatever.sas. Here is the text file for this example.
OPTION LINESIZE=72 PAGESIZE=66; DATA
COLOR; 1 20 36 PROC
PROBIT C=0.5; 
Comments: COLOR data set name. Data Specify output
Specify plot 
I am not interested in you (or me) becoming a SAS expert so you can use this as a template for your analyses. Just replace what is in black with your own names and data. Take out the blue C=0.5 line if not forced choice or change value if not 2AFC. Basically, this file tells SAS what the data are; how to calculate the observed probability; what procedure to run (probit) and how to display the results.
Upload this text file to the VAX if you didn't create it there. (I use "Fetch" on a Macintosh.) You can put it in your main directory or make a new one. To execute this file type: "SAS WHATEVER". Remember WHATEVER.SAS is the name of the text file. SAS runs the procedure and creates two files: WHATEVER.LOG and WHATEVER.LIS. The LOG file will contain any errors if you find you have a problem. The LIS file contains your results. Download this file and then you can look at it in an editing program. To look at it on the VAX, type "TYPE WHATEVER.LIS". (I don't know VMS so I can't help you much with editing on the VAX.)
The output in the LIS file contains a lot of stuff. Here is an excerpt:
GoodnessofFit Tests
NOTE: Since the chisquare
is small (p > 0.1000), fiducial limits will Probit
Model in Terms of Tolerance Distribution Probit
Procedure

Results Summary
Using the Quality Factor metric, the chisquare = 1.01. The degrees of freedom = 3. And the Prob > Chi_square is 0.80. When this is greater than 0.1, your results show an OK fit. h = 1.01/3 = 0.337 but the h is not used because the chisquare is low enough. The 50% level (corresponding to mu) is 23.7 with a standard deviation of 26.9. The fiducial limits calculated from this (see the table at the end of the output) ranges from 7.2  35.5.
Repeating this analysis using the compression ratio metric gives these results:
chisquare
= 0.66
degrees of freedom = 3
P>chisquare = .88 (greater than 0.1 is OK)
h = 0.66/3=0.22 (not used since chisquare is low)
mu(50% level) = 11.1, standard deviation = 8.0
Fiducial limits, 6.5  14.9
So you can see the fit a little better with the compression ratio but not much.
Why do both analyses?
Which one would you recommend?
Continue
on to 