OVERVIEW

In a between-subjects design the levels of the independent variable are represented by different participants. A one-way between-subjects design has just one independent variable. In the room color study described in earlier chapters, for example, the independent variable was the color of the room. The independent variable is said to have levels, which are usually referred to as treatment levels or treatment conditions. A level is the value of the independent variable for a given group. Our room color variable had two levels, red and blue. In the data file, these conditions would be coded arbitrarily as 1 and 2.

We are not limited to having only two levels for an independent variable. Theoretically, we could have a very large number. Practically, however, half a dozen or so levels are about as many as you will ordinarily see. In the example that we will use in this chapter, we will have five levels. Because this is a between-subjects design, each participant is studied under just one of these conditions and contributes just a single score to the data analysis.

A NUMERICAL EXAMPLE

The SAT is used by a wide range of colleges and universities as part of the application process for college admission. Assume that we are interested in the effect of preparation time on SAT performance.

A NUMERICAL EXAMPLE OF A THREE-WAY DESIGN

In Chapter 8 we added a second independent variable into a between-subjects design to generate a two-way factorial. At this point you probably realize that we are not limited to combining just two between-subjects independent variables in research designs (although we are still limiting ourselves to analyzing a single dependent variable). Theoretically, we could combine many such variables together, despite the fact that the complexities of such designs grow exponentially as the designs get more complex. It is possible to see in the research literature some ANOVA designs using five independent variables and a few using four variables; however, three-way designs are the common limit for most research questions. If you understand the logic of analyzing a three-way design, you can invoke the same strategies to handle those with four or five independent variables.

We will illustrate the general principles of a three-way between-subjects design by using the following simplified hypothetical example data set. Assume that we wish to measure citizen satisfaction with the public school system. We sample individuals representing three different political preferences: liberal, moderate, and conservative (coded 1, 2, and 3, respectively, in the data file). We also code for whether or not these individuals voted in last election (yes coded as 1 and no coded as 2), using voting as an indicator of political involvement.

HISTORICAL OVERVIEW

The SAS Web site provides a comprehensive history of the software and the company. Here is a synopsis of that information. SAS, an acronym for Statistical Analysis Software, is a set of statistical analysis procedures housed together within a large application. The idea for it was conceived by Anthony J. Barr, a graduate student at North Carolina State University, between 1962 and 1964. Barr collaborated with Jim Goodnight in 1968 to integrate regression and ANOVA procedures into the software. The project received a major boost in 1973 from the contribution of John P. Sall. Other participants in the early years included Caroll G. Perkins, Jolayne W. Service, and Jane T. Helwig. The SAS Institute was established in Raleigh, NC in 1976 when the first base SAS software was released. The company moved to its present location, Cary, NC, in 1980.

As is true for SPSS, the procedures it performs are driven by code (SPSS calls it syntax) that comprises its own command language. SAS began being used on mainframe computers several decades ago when the only way to instruct the software to perform the statistical analyses was by punching holes on computer cards via a card-reader machine and later by typing in this code on an otherwise blank screen. It should be noted that, unlike SPSS, the vast majority of current SAS users still prefer a code-driven interface.

A NUMERICAL EXAMPLE OF A COMPLEX MIXED DESIGN

The complex mixed design that we consider in this chapter contains one between-subjects factor (the gender of the participants) and two within-subjects factors (the color of toys and the type of toy). Over the course of several days, eight-year-old children were brought into a room containing toys. On some occasions, the toys were of the hands-on type (e.g., balls, beads, building blocks); sometimes these toys were yellow and sometimes they were blue. On other occasions, the toys were of the pretend type (e.g., dolls, stuffed animals, action figures, dress-up clothes); sometimes these toys were yellow and sometimes they were blue.

The children were placed in the room for sixty seconds each time. While they were in the room, the children were allowed to do anything they liked (within the bounds of reason and safety). Observers recorded the number of seconds each child interacted with toys of a particular category (e.g., yellow hands-on toys). For each child, the average number of seconds of such interaction comprised the dependent measure.

The data for this hypothetical study are shown in Figure 15.1. There were five girls (subid 1 through 5) and five boys (subid 6 through 10). Girl 3, for example, spent an average of seven seconds playing with yellow hands-on toys, twelve seconds playing with yellow pretend toys, three seconds playing with blue hands-on toys, and four seconds playing with blue pretend toys.

SCALES OF MEASUREMENT

In Chapter 1, we indicated that variables can take on different values. Here, we will deal with the process of associating values with variables and how to summarize the sets of values we obtain from that process. This will prove to be important in guiding us toward situations in which it is appropriate to use the ANOVA technique.

THE PROCESS OF MEASUREMENT

Measurement represents a set of rules informing us of how values are assigned to objects or events. A scale of measurement describes a specific set of rules. Psychology became intimately familiar with scales of measurement when S. S. Stevens, in 1946, wrote a response to a committee of the British Association for the Advancement of Science. That committee was dealing with the possibility of measuring sensory events, and the members could not agree on whether or not it was possible to perform such measurements. Because they had focused on the work that Stevens had done to measure loudness of sound, he felt that a response to the committee from him was appropriate. But instead of arguing with the committee, Stevens adopted a different strategy, presenting instead a theory of measurement. He elaborated on this theory in 1951 when, as the editor of the Handbook of Experimental Psychology, he gave prominent treatment to this topic in his lead chapter.

Stevens identified four scales in his theory: nominal, ordinal, interval, and ratio scales in that order.

A three-way within-subjects design is one in which the levels of three within-subjects variables are combined factorially. Given that it is a within-subjects design, we still require that the participants contribute a data point to each of the separate cells (each of the unique combinations of the levels of the independent variables) of the design. The logistics of running a three-way study can be challenging but, partly because the subjects are their own controls and partly because each within-subject effect of interest is associated with its own error term (thus partitioning the total within-subjects error into separate error components), such a design has a considerable amount of statistical power. The key is to find appropriate applications of this design that are relatively immune to carry-over contamination effects.

A NUMERICAL EXAMPLE OF A THREE-WAY WITHIN-SUBJECTS DESIGN

The hypothetical data set that will serve as our example uses three time-independent variables as shown in Figure 12.1. We are engaged in market research studying the product look of powdered laundry detergent. One independent variable is the color of the powder: it is colored white, sea green, or baby blue. The second independent variable is whether or not the powder contains granules (flecks of bright shiny additions that in reality contribute nothing to the cleaning power of the detergent). The third independent variable is the color of the box: It is colored either aqua or orange.

WHAT IS ANALYSIS OF VARIANCE?

Analysis of variance (ANOVA) is a statistical technique used to evaluate the size of the difference between sets of scores. For example, a group of researchers might wish to learn if the room color in which college students are asked to respond to questions assessing their mood can affect their reported mood. Students are randomly assigned to complete a mood inventory in one of two rooms. Random assignment, one of the hallmarks of experimental design, is used in an attempt to assure that there is no bias in who is placed into which group by making it equally likely that any one person could have been assigned to either group. One of the rooms is painted a soft shade of blue that was expected to exert a calming effect on the students; the other room is painted a bright red that was presumed to be more agitating than calming. Higher numerical scores on the mood inventory indicate a more relaxed mood. At the end of the study, we score the mood inventory for all participants.

The research question in this example is whether mood as indexed by the score on the mood inventory was affected by room color. To answer this question, we would want to compare the mood scores of the two groups. If the mood scores obtained in the blue room were higher overall than those obtained in the red room, we might be inclined to believe that room color influenced mood.

HISTORICAL OVERVIEW

The SPSS Web site provides a comprehensive history of the company and the software they have produced. Here is a synopsis of that information. SPSS, an acronym for Statistical Package for the Social Sciences, is a set of statistical analysis procedures housed together within a large application. The software was developed in 1968 by two doctoral students at Stanford (Norman H. Nie and Dale H. Brent) and one recent Stanford graduate (Hadlai Hull). Nie and Hull brought the software to the University of Chicago and published the first manual in 1970. They incorporated the company in Illinois in 1975.

SPSS began to be used on mainframe computers, the only kind of computer available in many organizations such as universities. A user would sit down at a computer terminal and type SPSS syntax (the programming language) onto computer cards via a card-reader machine, and in the early 1980s, onto a blank screen. The syntax would look like word strings but conformed to rules that enabled the software to perform the statistical analysis that was specified by the syntax.

SPSS was first marketed for personal computers in 1984. The revolutionary difference in that marketplace was the use of a graphical user interface (abbreviated GUI but thought of by most people today as using a mouse to point and click) to make selections from dialog screens. These selections were translated “behind the scenes” to SPSS syntax, but the syntax was not directly presented on the screen to the human user.

EXPERIMENTAL AND STATISTICAL CONTROL

The designs we have discussed to this point represent techniques of experimental control in order to reduce error variance in a research study (Kirk, 1995). It is also possible to reduce error variance by employing statistical control using analysis of covariance (ANCOVA). Statistical control is used when we know how participants stand (we know their score) on an additional variable that was not part of or could not readily be incorporated into the experimental design. This additional variable is brought into the data analysis by treating it as a covariate. Although covariates can be categorical or quantitative, we will limit our discussion and data analysis procedures to quantitatively measured covariates.

The covariate represents a potential source of variance that has not been experimentally controlled but could covary with the dependent variable and could therefore confound our interpretation of the results. By identifying that variable as a covariate and collecting measures of it on the study participants, it is possible to “remove” or “neutralize” its effect on the dependent variable (by statistical procedures described in this chapter) prior to analyzing the effects of the independent variable on the dependent variable.

A SIMPLE ILLUSTRATION OF COVARIANCE

Consider a test of math skills that we might administer to school children in a particular grade. We wish to compare girls and boys in their math problem solving skills.

THE CONCEPT OF WITHIN-SUBJECTS VARIANCE

In between-subjects designs, participants contribute a single value of the dependent variable, and different participants are represented under each level of the independent variable. The situation is different for within-subjects designs. Here, we register a value of the dependent variable in each and every condition of the study for every one of the participants or cases in the study. That is, each participant is measured under each level of the independent variable.

Within-subjects designs are still univariate studies in that there is only one dependent variable in the design. As is true for between-subjects designs, the dependent variable is operationalized in a particular manner. What differentiates between-subjects and within-subjects designs is the number of such measurements in the data file that are associated with each participant: In between-subjects designs, researchers record just one data point (measurement) for each participant; in within-subjects designs, researchers record as many measurements for each participant as there are conditions in the study.

Within-subjects designs can theoretically contain any number of independent variables combined in a factorial manner. Practically, because all participants must contribute data under each level of each independent variable, the logistics of the data collection procedures can become quite challenging with several independent variables in the design. Also, there is the possibility that subjects could be affected or changed by exposure to one level of an independent variable, a situation known as carry-over effects, such that it may not be appropriate to expose them to more than that single level; the likelihood of eliminating all of the possible carry-over effects diminishes with each additional independent variable included in the design (we will discuss carry-over effects more in Section 10.4).