Homework assignment.

A Statistical Experiment.

Due date: November 18, 2002

The purpose of this assignment is to create a vivid experience that illustrates some of the most important principles involved in conducting a statistical experiment. You will perform a version of the experiment that R.A. Fisher used in his book on experimental design.

1) Read the notes from class, November 1.

2) Select a beverage product that is produced by at least two companies. E.g. apple juice, cranberry juice, beer (if 21 or older), milk, cola (or other flavored soda), bottled water, etc. Go to the supermarket and select two brands of one product and purchase a container of each brand. For example, get a carton of Kleinpeter 2% milk and a carton of Borden 2% milk. In selecting your purchase, choose products that differ only in brand, not in kind. (Don't buy reconstituted orange juice fron one company and fresh orange juice from another.) You will test to see whether subjects can tell the difference between the brands.

3) Recruit 3 or more volunteers who are willing to taste samples.

4) Prepare 10 sample cups by labeling them 1--10. Assign the letter A to one of your brands and the letter B to the other. Write the letter A on one 8.5x11 sheet of paper and the letter B on another. For each subject, make a random key that assigns five As and five Bs to the numbers 1--10.

• I created a bank of 50 random keys for your convenience. To use this, open a book at random, and glance at the page number. Subtract 50 from the page number over and over again until a number between 0 and 49 is left. Select the corresponding key. (Example: I opened to page 182. After subtracting 50 three times, 32 is left. So, I take the key marked 32.)
• You may also use the random sequence generator at www.random.org. To use this site to generate the key, enter 1 as smallest value and 10 as largest value. The website will return to you the numbers 1 through 10 written in random order. You may assign A to the first five and B to the rest, or you may use some other scheme (such as alternating As and Bs) that you decide before retreiving the random sequence. Write the key on a piece of paper and retain it. (For example, if I use the "first five" rule and retrieve the sequence: 5, 8, 10, 1, 4, 6, 7, 3, 2, 9, then I write out on a slip of paper: "1-A, 2-B, 3-B, 4-A, 5-A, 6-B, 7-B, 8-A, 9-B, 10-A.")
• DO NOT attempt to "make up" a random key by yourself. People are not good at randomizing. What you suppose to be random may not be.

5) When you are ready to run the experiment, have your subject wait where he or she cannot see you. Fill the sample cups according to one of the keys prepared in step 4, then arrange them in order 1 through 10. Call your subject to perform the taste test. Explain to the subject that there are 5 cups of each brand. Allow the subject to taste the samples one at a time in order 1--10. After each tasting, the subject is to place the cup either on a sheet of paper marked A or the sheet marked B.

• Prior to the tasting, you must decide the precise rules and explain them to the subject. For example, you may require your subject to taste the samples in order and never allow going back to retaste earlier cups. Or you may allow the taster to go back and retaste earlier samples. If retasting is allowed, you may or may not allow revisions. (In class, I allowed retasting and revision at all times, as often as the taster wished.)
• Another variation that you could implement would be to place a sample cup of each brand on each sheet before the tasting begins, and allow the taster to refer to the samples.
• NOTE: I tend to think that the most interesting data would be obtained when the two brands had a difference that was barely perceptable to a person with a well-developed sense of taste. I would allow unlimited tastings, and would provide reference samples. BUT you may choose the rules for your tasting accoring to your best judgment.

6) When the taster is finished, record his/her classification on the key you used for filling the cups. Note how many cups were correctly classified as brand A. This will be a number from 1 to 5. Repeat for the other tasters.

7) Prepare a written report that includes:

• a description of the products you used;
• a description of the subjects;
• a description of the procedure you followed (see 4 and 5, above)
• a conclusion about the subjects ability to detect a difference between the products.

In giving you conclusion, state your level of confidence. This statement should have the following form: "If subjects who have no ability to distinguish participate in taste trials of the kind conducted here, then in the long run, one in every (so-and-so many) subjects will get a perfect score of 5, one in every (so-and-so many) subjects will get a score of 4 or better and one in every (so-and-so many) subjects will get a score of 3 or better. Thus, with confidence (such-and-such), subject 1 can distinguish; with confidence (such-and-such), subject 2 can distinguish; etc.

For extra credit, you may elaborate this experiment. For example, you may attempt to determine if subjects who cannot distinguish can be trained to distinguish. Or you may conduct the experiment with 12 cups rather than 10. Or you may propose a variant and submit it to me for comment. Extra credit work on this experiment will be accepted until the last day of classes, but not afterwards.