The purpose of this final chapter is to provide a brief introduction to some selected topics related to experimental design and ANOVA procedures. These topics include interaction comparisons, random and fixed factors, nested factors, Latin squares, unequal sample sizes, and multivariate analysis of variance. Because complete coverage of these topics requires at least a separate chapter per topic, which is beyond the scope of the present book, our coverage will be somewhat cursory; sources that may be consulted for further information are provided in connection with each topic.

INTERACTION COMPARISONS

SIMPLE EFFECTS ANALYSES

As we indicated in Chapter 8, most researchers explore a statistically significant A × B interaction effect by conducting simple effects analyses. The simple effects strategy that we have used throughout this book was to perform pairwise comparisons using t tests directly following the omnibus ANOVA that yielded a statistically significant interaction effect. An alternative but similar strategy with three or more levels of one of the independent variables can be illustrated by considering the means displayed in Figure 17.1. This 3 × 2 factorial was originally presented in Figure 8.2. In this alternative but similar strategy, we focus on one level of one of the independent variables at a time. In Figure 17.1 we have outlined the means of the females to highlight one focus, and would repeat this focus with the males. To implement this strategy, we would do the following:

1. Perform a one-way ANOVA for the females comparing the means of type of residence.

2. […]

OVERVIEW

In a between-subjects design the levels of the independent variable are represented by different participants. A one-way between-subjects design has just one independent variable. In the room color study described in earlier chapters, for example, the independent variable was the color of the room. The independent variable is said to have levels, which are usually referred to as treatment levels or treatment conditions. A level is the value of the independent variable for a given group. Our room color variable had two levels, red and blue. In the data file, these conditions would be coded arbitrarily as 1 and 2.

We are not limited to having only two levels for an independent variable. Theoretically, we could have a very large number. Practically, however, half a dozen or so levels are about as many as you will ordinarily see. In the example that we will use in this chapter, we will have five levels. Because this is a between-subjects design, each participant is studied under just one of these conditions and contributes just a single score to the data analysis.

A NUMERICAL EXAMPLE

The SAT is used by a wide range of colleges and universities as part of the application process for college admission. Assume that we are interested in the effect of preparation time on SAT performance.

A NUMERICAL EXAMPLE OF A THREE-WAY DESIGN

In Chapter 8 we added a second independent variable into a between-subjects design to generate a two-way factorial. At this point you probably realize that we are not limited to combining just two between-subjects independent variables in research designs (although we are still limiting ourselves to analyzing a single dependent variable). Theoretically, we could combine many such variables together, despite the fact that the complexities of such designs grow exponentially as the designs get more complex. It is possible to see in the research literature some ANOVA designs using five independent variables and a few using four variables; however, three-way designs are the common limit for most research questions. If you understand the logic of analyzing a three-way design, you can invoke the same strategies to handle those with four or five independent variables.

We will illustrate the general principles of a three-way between-subjects design by using the following simplified hypothetical example data set. Assume that we wish to measure citizen satisfaction with the public school system. We sample individuals representing three different political preferences: liberal, moderate, and conservative (coded 1, 2, and 3, respectively, in the data file). We also code for whether or not these individuals voted in last election (yes coded as 1 and no coded as 2), using voting as an indicator of political involvement.

HISTORICAL OVERVIEW

The SAS Web site provides a comprehensive history of the software and the company. Here is a synopsis of that information. SAS, an acronym for Statistical Analysis Software, is a set of statistical analysis procedures housed together within a large application. The idea for it was conceived by Anthony J. Barr, a graduate student at North Carolina State University, between 1962 and 1964. Barr collaborated with Jim Goodnight in 1968 to integrate regression and ANOVA procedures into the software. The project received a major boost in 1973 from the contribution of John P. Sall. Other participants in the early years included Caroll G. Perkins, Jolayne W. Service, and Jane T. Helwig. The SAS Institute was established in Raleigh, NC in 1976 when the first base SAS software was released. The company moved to its present location, Cary, NC, in 1980.

As is true for SPSS, the procedures it performs are driven by code (SPSS calls it syntax) that comprises its own command language. SAS began being used on mainframe computers several decades ago when the only way to instruct the software to perform the statistical analyses was by punching holes on computer cards via a card-reader machine and later by typing in this code on an otherwise blank screen. It should be noted that, unlike SPSS, the vast majority of current SAS users still prefer a code-driven interface.

A NUMERICAL EXAMPLE OF A COMPLEX MIXED DESIGN

The complex mixed design that we consider in this chapter contains one between-subjects factor (the gender of the participants) and two within-subjects factors (the color of toys and the type of toy). Over the course of several days, eight-year-old children were brought into a room containing toys. On some occasions, the toys were of the hands-on type (e.g., balls, beads, building blocks); sometimes these toys were yellow and sometimes they were blue. On other occasions, the toys were of the pretend type (e.g., dolls, stuffed animals, action figures, dress-up clothes); sometimes these toys were yellow and sometimes they were blue.

The children were placed in the room for sixty seconds each time. While they were in the room, the children were allowed to do anything they liked (within the bounds of reason and safety). Observers recorded the number of seconds each child interacted with toys of a particular category (e.g., yellow hands-on toys). For each child, the average number of seconds of such interaction comprised the dependent measure.

The data for this hypothetical study are shown in Figure 15.1. There were five girls (subid 1 through 5) and five boys (subid 6 through 10). Girl 3, for example, spent an average of seven seconds playing with yellow hands-on toys, twelve seconds playing with yellow pretend toys, three seconds playing with blue hands-on toys, and four seconds playing with blue pretend toys.

SCALES OF MEASUREMENT

In Chapter 1, we indicated that variables can take on different values. Here, we will deal with the process of associating values with variables and how to summarize the sets of values we obtain from that process. This will prove to be important in guiding us toward situations in which it is appropriate to use the ANOVA technique.

THE PROCESS OF MEASUREMENT

Measurement represents a set of rules informing us of how values are assigned to objects or events. A scale of measurement describes a specific set of rules. Psychology became intimately familiar with scales of measurement when S. S. Stevens, in 1946, wrote a response to a committee of the British Association for the Advancement of Science. That committee was dealing with the possibility of measuring sensory events, and the members could not agree on whether or not it was possible to perform such measurements. Because they had focused on the work that Stevens had done to measure loudness of sound, he felt that a response to the committee from him was appropriate. But instead of arguing with the committee, Stevens adopted a different strategy, presenting instead a theory of measurement. He elaborated on this theory in 1951 when, as the editor of the Handbook of Experimental Psychology, he gave prominent treatment to this topic in his lead chapter.

Stevens identified four scales in his theory: nominal, ordinal, interval, and ratio scales in that order.

A three-way within-subjects design is one in which the levels of three within-subjects variables are combined factorially. Given that it is a within-subjects design, we still require that the participants contribute a data point to each of the separate cells (each of the unique combinations of the levels of the independent variables) of the design. The logistics of running a three-way study can be challenging but, partly because the subjects are their own controls and partly because each within-subject effect of interest is associated with its own error term (thus partitioning the total within-subjects error into separate error components), such a design has a considerable amount of statistical power. The key is to find appropriate applications of this design that are relatively immune to carry-over contamination effects.

A NUMERICAL EXAMPLE OF A THREE-WAY WITHIN-SUBJECTS DESIGN

The hypothetical data set that will serve as our example uses three time-independent variables as shown in Figure 12.1. We are engaged in market research studying the product look of powdered laundry detergent. One independent variable is the color of the powder: it is colored white, sea green, or baby blue. The second independent variable is whether or not the powder contains granules (flecks of bright shiny additions that in reality contribute nothing to the cleaning power of the detergent). The third independent variable is the color of the box: It is colored either aqua or orange.

WHAT IS ANALYSIS OF VARIANCE?

Analysis of variance (ANOVA) is a statistical technique used to evaluate the size of the difference between sets of scores. For example, a group of researchers might wish to learn if the room color in which college students are asked to respond to questions assessing their mood can affect their reported mood. Students are randomly assigned to complete a mood inventory in one of two rooms. Random assignment, one of the hallmarks of experimental design, is used in an attempt to assure that there is no bias in who is placed into which group by making it equally likely that any one person could have been assigned to either group. One of the rooms is painted a soft shade of blue that was expected to exert a calming effect on the students; the other room is painted a bright red that was presumed to be more agitating than calming. Higher numerical scores on the mood inventory indicate a more relaxed mood. At the end of the study, we score the mood inventory for all participants.

The research question in this example is whether mood as indexed by the score on the mood inventory was affected by room color. To answer this question, we would want to compare the mood scores of the two groups. If the mood scores obtained in the blue room were higher overall than those obtained in the red room, we might be inclined to believe that room color influenced mood.

HISTORICAL OVERVIEW

The SPSS Web site provides a comprehensive history of the company and the software they have produced. Here is a synopsis of that information. SPSS, an acronym for Statistical Package for the Social Sciences, is a set of statistical analysis procedures housed together within a large application. The software was developed in 1968 by two doctoral students at Stanford (Norman H. Nie and Dale H. Brent) and one recent Stanford graduate (Hadlai Hull). Nie and Hull brought the software to the University of Chicago and published the first manual in 1970. They incorporated the company in Illinois in 1975.

SPSS began to be used on mainframe computers, the only kind of computer available in many organizations such as universities. A user would sit down at a computer terminal and type SPSS syntax (the programming language) onto computer cards via a card-reader machine, and in the early 1980s, onto a blank screen. The syntax would look like word strings but conformed to rules that enabled the software to perform the statistical analysis that was specified by the syntax.

SPSS was first marketed for personal computers in 1984. The revolutionary difference in that marketplace was the use of a graphical user interface (abbreviated GUI but thought of by most people today as using a mouse to point and click) to make selections from dialog screens. These selections were translated “behind the scenes” to SPSS syntax, but the syntax was not directly presented on the screen to the human user.