II. Surveys, Observational Studies, and Experiments

Know what is required for a sample to be a simple random sample (SRS). If each individual in the population has an equal probability of being chosen for a sample, it doesn't follow that the sample is an SRS. Consider a class of six boys and six girls. I want to randomly pick a committee of two students from this group. I decide to flip a coin. If "heads," I will choose two girls by a random process. If "tails," I will choose two boys by a random process. Now, each student has an equal probability (1/6) of being chosen for the committee. However, the two students are not an SRS of size two picked from members of the class. Why not? Because this selection process does not allow for a committee consisting of one boy and one girl. To have an SRS of size two from the class, each group of two students would have to have an equal probability of being chosen as the committee.

SRS refers to how you obtain your sample; random allocation is what you use in an experiment to assign subjects to treatment groups. They are not synonyms.

Well-designed experiments satisfy the principles of control, randomization, and replication.

    * Control for the effects of lurking variables by comparing several treatments in the same environment. Note: Control is not synonymous with "control group."

    * Randomization refers to the random allocation of subjects to treatment groups, and not to the selection of subjects for the experiment. Randomization is an attempt to "even out" the effects of lurking variables across the treatment groups. Randomization helps avoid bias.

    * Replication means using a large enough number of subjects to reduce chance variation in a study.

      Note: In science, replication often means, "do the experiment again."

Distinguish the language of surveys from the language of experiments.      Stratifying:sampling::Blocking:experiment

It is not enough to memorize the terminology related to surveys, observational studies, and experiments. You must be able to apply the terminology in context. For example:

Blocking refers to a deliberate grouping of subjects in an experiment based on a characteristic (such as gender, cholesterol level, race, or age) that you suspect will affect responses to treatments in a systematic way. After blocking, you should randomly assign subjects to treatments within the blocks. Blocking reduces unwanted variability.

An experiment is double blind if neither the subjects nor the experimenters know who is receiving what treatment. A third party can keep track of this information.

Suppose that subjects in an observational study who eat an apple a day get significantly fewer cavities than subjects who eat less than one apple each week. A possible confounding variable is overall diet. Members of the apple-a-day group may tend to eat fewer sweets, while those in the non-apple-eating group may turn to sweets as a regular alternative. Since different diets could contribute to the disparity in cavities between the two groups, we cannot say that eating an apple each day causes a reduction in cavities.