COMBINING BETWEEN- AND WITHIN-SUBJECTS FACTORS

As discussed in Chapter 13, a mixed design is one that contains at least one between-subjects independent variable and at least one within-subjects independent variable. Simple mixed designs have only two independent variables and so, by definition, must have one of each type of variable. Complex mixed designs contain at least three independent variables. The three-way complex mixed designs presented in this chapter and in Chapter 15 must (by definition) have two of one type and one of the other type of factor. In this chapter, we will focus on the design with two between-subjects factors and one within-subjects factor; Chapter 15 will present the design with one between-subjects factor and two within-subjects factors.

A NUMERICAL EXAMPLE OF A COMPLEX MIXED DESIGN

College students who signed up for a research study read vignettes in which their romantic partner was described as being attracted to another person; this attraction was depicted as being either emotional attraction with no physical component or physical attraction with no emotional component. Because this type-of-attraction variable was intended to be a within-subjects variable, students read both vignettes. After reading each vignette, the students completed a short inventory evaluating their feelings of jealousy; the response to this inventory served as the dependent variable. For the purposes of this hypothetical example, assume that no effects concerning the order of reading these vignettes was obtained; thus, we will present the results without considering the vignette ordering factor.

OVERVIEW

Once we have determined that an independent variable has yielded a significant effect, we must next turn our attention to the differences between the means of the conditions in the study. If there are only two means, then we automatically know that they are significantly different. With three or more means a significant F ratio reveals only that there is a difference between at least one pair of means in the design; in this case we must perform an additional, post-ANOVA multiple comparison procedure to determine which of the three or more means differ from which others.

There are a variety of multiple comparison procedures that are available to researchers. Before presenting them, we will first describe some dimensions along which they differ; this will help us, as we go through this chapter, to discuss the differences among them.

PLANNED VERSUS UNPLANNED COMPARISONS

In an idealized world of research, hypotheses regarding differences between certain of the groups in a design are formulated in advance of the data collection on the basis of the theoretical context out of which the research was generated. Once the study is completed and a statistically significant F ratio is obtained in the omnibus ANOVA (although in this idealized world a statistically significant F ratio may not even be necessary if there are a very few hypothesized mean differences), the researchers then carry out the mean comparisons that they have already specified.

COMBINING BETWEEN-SUBJECTS AND WITHIN-SUBJECTS FACTORS

A mixed design is one that contains at least one between-subjects independent variable and at least one within-subjects independent variable. In a simple mixed design, there are only two independent variables, one a between-subjects factor and the other a within-subjects factor; these variables are combined factorially. The number of levels of each independent variable is not constrained by the design. Thus, we could have a 2 × 2, a 4 × 3, or even a 3 × 7 factorial design. Chapters 14 and 15 will address two complex mixed designs that contain three independent variables.

Because there are two independent variables, there are three effects of interest: the main effect of the between-subjects variable (A), the main effect of the within-subjects variable (B), and the two-way interaction (A × B). Note that this is analogous to what we have seen in the two-way between-subjects and two-way within-subjects designs. Furthermore, the conceptual understanding of main effects and interactions in those designs carries forward to the simple mixed design. Main effects focus on the mean differences of the levels of each independent variable (e.g., a1 vs. a2) and interactions focus on whether or not the patterns of differences are parallel (e.g., a1 vs. a2 under b1 compared to a1 vs. a2 under b2).

The primary difference between a simple mixed design and the between-subjects and within-subjects designs is in the way that the total variance of the dependent variable is partitioned.

COMBINING TWO INDEPENDENT VARIABLES FACTORIALLY

NAMING THE DESIGN

In Chapter 6, we discussed a between-subjects design that contained a single independent variable (preparation time for the SAT). However, we are not limited to studying the effects of just one independent variable in a research design. In this chapter, we will deal with the inclusion of a second independent variable (note that we still have only one dependent variable). A design containing more than one independent variable is known as a factorial design when the variables are combined in a manner described in Section 8.1.2. When those independent variables are between-subjects variables, the design is called a between-subjects design or a between-subjects factorial design. Designs containing two between-subjects independent variables that are simultaneously varied are two-way between-subjects (factorial) designs. These designs are also sometimes referred to as two-way completely randomized designs because subjects are assumed to be randomly assigned to the various treatments.

COMBINING INDEPENDENT VARIABLES IN A SINGLE DESIGN

Intertwining two independent variables within the same design is done by combining them in a factorial fashion in which each level of one independent variable is combined with each level of the other independent variable. If our independent variables were, for example, gender (female and male) and size of city in which participants resided (large and small), then one combination of the levels of the independent variables might be females living in large cities. This would then be one condition or group in the factorial design.

The purpose of this final chapter is to provide a brief introduction to some selected topics related to experimental design and ANOVA procedures. These topics include interaction comparisons, random and fixed factors, nested factors, Latin squares, unequal sample sizes, and multivariate analysis of variance. Because complete coverage of these topics requires at least a separate chapter per topic, which is beyond the scope of the present book, our coverage will be somewhat cursory; sources that may be consulted for further information are provided in connection with each topic.

INTERACTION COMPARISONS

SIMPLE EFFECTS ANALYSES

As we indicated in Chapter 8, most researchers explore a statistically significant A × B interaction effect by conducting simple effects analyses. The simple effects strategy that we have used throughout this book was to perform pairwise comparisons using t tests directly following the omnibus ANOVA that yielded a statistically significant interaction effect. An alternative but similar strategy with three or more levels of one of the independent variables can be illustrated by considering the means displayed in Figure 17.1. This 3 × 2 factorial was originally presented in Figure 8.2. In this alternative but similar strategy, we focus on one level of one of the independent variables at a time. In Figure 17.1 we have outlined the means of the females to highlight one focus, and would repeat this focus with the males. To implement this strategy, we would do the following:

1. Perform a one-way ANOVA for the females comparing the means of type of residence.

2. […]

OVERVIEW

In a between-subjects design the levels of the independent variable are represented by different participants. A one-way between-subjects design has just one independent variable. In the room color study described in earlier chapters, for example, the independent variable was the color of the room. The independent variable is said to have levels, which are usually referred to as treatment levels or treatment conditions. A level is the value of the independent variable for a given group. Our room color variable had two levels, red and blue. In the data file, these conditions would be coded arbitrarily as 1 and 2.

We are not limited to having only two levels for an independent variable. Theoretically, we could have a very large number. Practically, however, half a dozen or so levels are about as many as you will ordinarily see. In the example that we will use in this chapter, we will have five levels. Because this is a between-subjects design, each participant is studied under just one of these conditions and contributes just a single score to the data analysis.

A NUMERICAL EXAMPLE

The SAT is used by a wide range of colleges and universities as part of the application process for college admission. Assume that we are interested in the effect of preparation time on SAT performance.

A NUMERICAL EXAMPLE OF A THREE-WAY DESIGN

In Chapter 8 we added a second independent variable into a between-subjects design to generate a two-way factorial. At this point you probably realize that we are not limited to combining just two between-subjects independent variables in research designs (although we are still limiting ourselves to analyzing a single dependent variable). Theoretically, we could combine many such variables together, despite the fact that the complexities of such designs grow exponentially as the designs get more complex. It is possible to see in the research literature some ANOVA designs using five independent variables and a few using four variables; however, three-way designs are the common limit for most research questions. If you understand the logic of analyzing a three-way design, you can invoke the same strategies to handle those with four or five independent variables.

We will illustrate the general principles of a three-way between-subjects design by using the following simplified hypothetical example data set. Assume that we wish to measure citizen satisfaction with the public school system. We sample individuals representing three different political preferences: liberal, moderate, and conservative (coded 1, 2, and 3, respectively, in the data file). We also code for whether or not these individuals voted in last election (yes coded as 1 and no coded as 2), using voting as an indicator of political involvement.

HISTORICAL OVERVIEW

The SAS Web site provides a comprehensive history of the software and the company. Here is a synopsis of that information. SAS, an acronym for Statistical Analysis Software, is a set of statistical analysis procedures housed together within a large application. The idea for it was conceived by Anthony J. Barr, a graduate student at North Carolina State University, between 1962 and 1964. Barr collaborated with Jim Goodnight in 1968 to integrate regression and ANOVA procedures into the software. The project received a major boost in 1973 from the contribution of John P. Sall. Other participants in the early years included Caroll G. Perkins, Jolayne W. Service, and Jane T. Helwig. The SAS Institute was established in Raleigh, NC in 1976 when the first base SAS software was released. The company moved to its present location, Cary, NC, in 1980.

As is true for SPSS, the procedures it performs are driven by code (SPSS calls it syntax) that comprises its own command language. SAS began being used on mainframe computers several decades ago when the only way to instruct the software to perform the statistical analyses was by punching holes on computer cards via a card-reader machine and later by typing in this code on an otherwise blank screen. It should be noted that, unlike SPSS, the vast majority of current SAS users still prefer a code-driven interface.

A NUMERICAL EXAMPLE OF A COMPLEX MIXED DESIGN

The complex mixed design that we consider in this chapter contains one between-subjects factor (the gender of the participants) and two within-subjects factors (the color of toys and the type of toy). Over the course of several days, eight-year-old children were brought into a room containing toys. On some occasions, the toys were of the hands-on type (e.g., balls, beads, building blocks); sometimes these toys were yellow and sometimes they were blue. On other occasions, the toys were of the pretend type (e.g., dolls, stuffed animals, action figures, dress-up clothes); sometimes these toys were yellow and sometimes they were blue.

The children were placed in the room for sixty seconds each time. While they were in the room, the children were allowed to do anything they liked (within the bounds of reason and safety). Observers recorded the number of seconds each child interacted with toys of a particular category (e.g., yellow hands-on toys). For each child, the average number of seconds of such interaction comprised the dependent measure.

The data for this hypothetical study are shown in Figure 15.1. There were five girls (subid 1 through 5) and five boys (subid 6 through 10). Girl 3, for example, spent an average of seven seconds playing with yellow hands-on toys, twelve seconds playing with yellow pretend toys, three seconds playing with blue hands-on toys, and four seconds playing with blue pretend toys.