2.1. Team output can be influenced by team dynamics and interaction

Teams are a complex and dynamic system as its members adapt to the various situations and demands (Kozlowski & Ilgen, 2006). The benefits of the team in a project setting have been proven with numerous previous research (Dennis, Reference Dennis1996; Gyory et al., Reference Gyory, Cagan and Kotovsky2019; Karau & Williams, 1993). For example, good team dynamic can improve team project outcome (DiTullio, 2010), while negative team dynamics and experience can damage performance and outcomes (Buckenmyer, Reference Buckenmyer2000; Jones, Reference Jones1996). When asked, students often indicated that things like bad interaction and organization, bad communication, and bad collaboration contributed to their bad experiences (Buckenmyer, Reference Buckenmyer2000). To investigate more closely at teamwork, some factors from Salas et al.'s Nine Critical Considerations of Teamwork (9 C's) (2015) as well as the factors of cohesiveness and creativity were used to set the structure of the examination.

The first factor of the model is team composition, which can have a complicated effect on the team performance. For example, more diverse teams have been found to perform better because members are able to bring different contributions (De Cooman et al., 2015; Muchinsky & Monahan, 1987; Neuman et al., Reference Neuman, Wagner and Christiansen1999; Piasentin & Chapman, 2007). On the other hand, more homogeneous teams have also been found to perform better as similarities between members foster a sense of belonging (Muchinsky & Monahan, 1987; Neuman et al., Reference Neuman, Wagner and Christiansen1999). Another factor is communication, where effective communication is positively linked to member satisfaction and motivation (Brewer et al., Reference Brewer, Mitchell, Sanders, Wallace and Wood2015; Mikkelson et al., Reference Mikkelson, York and Arritola2015). This could be due to communication builds trust and connection which can improve teamwork (Zemliansky, Reference Zemliansky2012). Conflict is another factor. And contrary to popular belief that conflict impedes team cohesion and negatively impacts emotion (Gladstein, Reference Gladstein1984; Hackman & Morris, 1975; Saavedra et al., Reference Saavedra, Earley and Dyne1993; J. A. Wall Jr & Callister, 1995; V. D. Wall Jr & Nolan, 1986), depending on the circumstances, conflict can have positive effects as well. For example, it was found that some conflicts can stimulate the exchange of information during the decision-making process (Levine et al., Reference Levine, Resnick and Higgins1993), become more creative and boost their problem-solving skills (Levine et al., Reference Levine, Resnick and Higgins1993; Nemeth, Reference Nemeth1986; Tjosvold, Reference Tjosvold1997). Coordination as a factor is tied to communication, where good coordination involves the members of the team sharing a common mental model (Cannon-Bowers et al., 1990; Orasanu, Reference Orasanu2022). This shared mental model enables members to make similar decisions and actions without explicit communication (E. E. Entin & Serfaty, 1999). This coordination was then found to be positively related to team performance. (E. Entin et al., Reference Entin, Serfaty, Entin and Deckert1993), and can help to decrease the cognitive load in stressful situations, as members understand each other and what needs to be done (Orasanu, Reference Orasanu2022). The factor of cooperation is when team members emphasize accomplishment as a group and collaboration (Beersma et al., Reference Beersma, Hollenbeck, Humphrey, Moon, Conlon and Ilgen2003). These traits positively influence team performance and output, as collaboration can build trust and cohesiveness between team members (Ivancevich et al., Reference Ivancevich, Matteson and Konopaske1990). Group cohesion, which is defined as the willingness of the group to prefer being with each other to complete the overall goal (Carron et al., Reference Carron, Brawley and Widmeyer1998), has been found to significantly influence team performance (Carron et al., Reference Carron, Colman, Wheeler and Stevens2002; Heuzé et al., 2006). This relationship was further supported in sports-related team research, where it was found that teams with higher levels of cohesion had a higher success rate (Martens & Peterson, 1971). Finally, good teamwork work has often been found to promote creativity in the designs they produce. For example, a study found that increased diversity in teams has also been found to increase their innovative rating assessment scores (Ancona & Caldwell, 1992). And good team environment can also help to decrease anxiety associated with producing and selecting creative ideas and increasing psychological safety (A. C. Edmondson, Reference Edmondson, Kramer and Cook2004; Shin & Eom, 2014).

2.2. Evolution of psychological safety in teams

From previous literature, it could be seen that many factors impacting team experience and output are linked to the individual's perception of psychological safety in teams. For example, higher psychological safety can help individuals to more confidently express their opinions and increase their creativity (A. C. Edmondson, Reference Edmondson, Kramer and Cook2004). In addition, higher psychological safety can also help individuals to provide feedback to their peers and engage in meaningful discussions (A. C. Edmondson, Reference Edmondson2003; West, Reference West1990). And with the presence of conflict in a team related to the task or project problem, the presence and level of psychological safety within the team can influence whether the conflict will have a positive or negative impact on team performance (Bradley et al., Reference Bradley, Postlethwaite, Klotz, Hamdani and Brown2012).

Because of the importance of psychological safety, it is also important to understand how it can be built. The first area to investigate is the factors impacting team performance, which are related to psychological safety. More specifically, the factors of interpersonal relationships, group interaction and dynamics, the leadership within the group, and the organization's culture and norms are all related to the development of psychological safety (Kahn, Reference Kahn1990).

Time as a factor might also impact psychological safety, as it takes time to develop (Kozlowski et al., Reference Kozlowski, Chao, Grand, Braun and Kuljanin2013). More specifically, in the paper by Cole et al. (2022) examining psychological safety in freshman engineering students, they found that although psychological safety can be reliably measured across a continuous time, it will tend to be more unreliable toward the beginning. In the same paper, the authors found that psychological safety perceptions of the team decreased, which might be an indicator that the variation of psychological safety perception of the individual team members increased (Cole, O’Connell, Gong, et al., 2022). The stage of the design process has also been found to impact psychological safety development in freshman engineering students, with concept generation and selection having the lowest psychological safety scores (Miller et al., Reference Miller, Marhefka, Heininger, Jablokow, Mohammed and Ritter2019). Of the seven C factors (Composition, communication, coordination, cooperation, cohesiveness, creativity, and conflict) they investigated, communication and cohesiveness were found to impact the team's performance and output significantly (Cole, O’Connell, Gong, et al., 2022). However, although these studies have provided valuable findings, the population they focused on was mainly freshman engineering students.

3.1. Participants

Participants for this study were all undergraduate students who enrolled in a fourth-year senior capstone engineering design course at a large northeastern university in the Spring semester of 2023 and were 18 years or older. Participants were recruited in their classrooms after consent was initially gathered from their instructors via email. A total of 26 participants were recruited, but those with missing data points were removed from the analysis, resulting in the final 18 that were analyzed for this paper. Due to the focus of the study being the psychological safety scores, no demographic data were collected from the participants.

3.2. Procedure

The data for this study was completed in a fourth-year capstone engineering design course, and the data collection process followed that of the one used in a previous study by Cole et al., (2022), with adaptations for use in a capstone course. Approval was obtained from the Institutional Review Board (IRB) at the Pennsylvania State University. Researchers obtained consent from the instructors to use some class time to collect the data. Data collection occurred at five time points over one spring semester in 2023: 1) start of the project, 2) idea generation, 3) concept selection, 4) prototyping, and 5) final design deliverables. The process of data collection can be better visualized in Figure 1. More specifically, the time between each time point could range from one to five class sessions (equating to 1-3 weeks, given the 2/week class meetings), depending on the curriculum progress and the instructor's needs. Each design session occurred within one class period. Details on what student participants were completing in the course curriculum can be better found in Cole et al. (2022), they will only briefly be explained here.

Figure 1. Data collection process

For time point 1, participants were assigned to groups of three to five depending on their project. The participants then researched the project they were completing. For time point 2, students were guided through the process of idea generation and were asked to generate solutions to the problem they are working on. At time point 3, teams were guided through the process of concept screening and were asked to evaluate the ideas their team members generated. At time point 4, the students were asked to build prototypes of the ideas that passed concept screening. Finally, at time point 5, teams presented their final ideas to the class using PowerPoint presentation and a fully constructed final prototype. For all time points, the survey was completed at the end of the class.

More specifically, at each time point, after the students had completed the assigned activity of interest, a survey was given to them to complete through Qualtrics. At the beginning of the survey, the students were prompted to create a unique code, which they used for all the five time points of data collection. The survey included seven questions developed by Edmondson (1999) to measure the psychological safety of the individual, and two open-ended questions. More specifically, the first seven questions collected 7-point Likert scale responses ranging from 1 (very inaccurate) to 7 (very accurate), and are aimed at examining how comfortable the individuals felt in the team in times when, for example, they made a mistake or if they felt accepted by the team (A. Edmondson, Reference Edmondson1999). For the score of psychological safety perception of everyone, it was calculated by first reverse coding any of the negative items in the survey, and then taking the average of all their responses. The two open-ended questions that followed the survey questions asked the participants to write two instances during that measured period where they experienced a positive and a negative interaction in the team.

3.3. Content analysis

To analyse the two open-ended responses captured as part of this survey, a qualitative analysis was conducted using NVIVO. More specifically, the questions asked the students to “Please describe any positive/negative team interactions or activities that impacted the rating.” For this part of the analysis, the code book used in the previous work by Cole et al., (2022) was used. This codebook was named the “Seven Critical Considerations (7C's) of Psychological Safety in Engineering Design Teamwork.” This structure was established by adding two additional factors (Creativity and Cohesiveness) (Mullen & Copper, 1994) to the five of the Critical Considerations for Teamwork Framework (Composition, Communication, Coordination, Cooperation, and Conflict) (Salas et al., Reference Salas, Shuffler, Thayer, Bedwell and Lazzara2015). For this part of the analysis, a total of 104 responses were collected between the students for four-time points, starting at Time Point 2. Of the 104 responses, those who did not give consent were removed, as well as those whose responses were blank, “none”, or “n/a.” Interrater reliability was established between two raters. One rater was a Post-Doctoral Scholar in Engineering Design and Innovation, and the other rater was a Ph.D. student in Industrial Engineering. The two raters independently coded 20% of the dataset here in NVIVO. They coded at the sentence level, and each sentence was allowed to be coded into multiple categories. Their inter-rater reliability (Cohen's Kappa) was checked at the end. When the reliability was below 0.75, they discussed the coding structure until they reached an agreement. They then went back to the text and re-coded the text. Finally, an inter-rater reliability (Cohen's Kappa) of 0.89% was achieved. Then, one coder coded the rest of this dataset.

For the coding structure, the first code, Cohesiveness, included the instances when participants mentioned anything regarding the team dynamic and team chemistry. For example, one participant mentioned “By this point in the semester, I have grown stronger relationships with certain team members.” This was coded to the Cohesiveness factor. For Communication, it was whenever the participants mentioned communication explicitly, or if they mentioned verbal interactions, such as “When we had a problem, all of us could talk about it and tried to figure it out.” The Composition factor encompassed descriptions of the team members, for example, “The team seems very knowledgeable in their respective fields.” The conflict factor was coded for all the times the participants mentioned things like agreement or disagreement between the teammates, such as “It's difficult to ensure that everyone is on the same page because of their different viewpoints and knowledge on the project thus far.” Next, anything that mentioned teamwork or working together as a team was coded to Cooperation, such as “One positive team interaction I had was working on the final report collaboratively.” For the factor coordination, it was whenever the participants mentioned anything related to assigning work, managing work or the team, etc. For example, “The team is usually on the same page with what the top priorities are.” was coded as Coordination for positive experience. Finally, creativity was when the participants mentioned ideation or selection of ideas, such as “Recently being able to show a “final” device to our sponsor and volunteers. Showed some light at the end of a long and tiring tunnel.”

3.4. Change in psychological safety

To investigate the building and waning of psychological safety in engineering design teams studied here, change in psychological safety was used. More specifically, change in psychological safety was calculated based on the psychological safety score of the individual at each time point. Their psychological safety score was calculated as average of their answers for the seven, 7-point Likert Scale type questions of psychological safety. The change in psychological safety was calculated by using this equation (Equation 1):

(1) $$\triangle \tx Psychological\;Safety_{ij} = (Psychological\;Safety)_i - (Psychological\;Safety)_j $$

where i is the current time point, ranging from 2-5 and j is the previous time point, ranging from 1-4. This process produces four changes in psychological safety scores for each participant and is either a negative value, zero, or positive value. A negative value indicates that the current time point score was lower than the previous time point score, and therefore shows that psychological safety decreased; 0 means that there was no change; and a positive value indicates that the psychological safety increased between the two time points.

4.1. RQ1: How did the psychological safety perception of the individual change with time?

This research question was established to explore if over the course of the semester the psychological safety perception of the students changed significantly. We hypothesize that time would have a significant impact on the development of psychological safety, as supported by prior research by (Cole, O’Connell, Gong, et al., 2022). However, as this is more explorative, there is no hypothesis regarding the specific relationship between time and psychological safety development for this population.

To answer this research question, a one-way repeated measures ANOVA was conducted, with the continuous dependent variable being the psychological safety score at each time point. Before conducting the actual analysis, assumptions were checked and were found to be satisfied. One outlier was found per inspection of the box plot. Analysis was done with and without the outlier, and no significant differences in results were observed, therefore, the outlier was kept in the final analysis. There were no statistically significant changes in the psychological safety perception of the students over time, F(4, 68) = 1.209, p = 0.315, partial = 0.066. Psychological safety perception of the students decreased from 5.603 ± 0.206 to 5.333 ± 0.248 at time point 2, then decreasing further to 5.258 ± 0.300 at time point 3, then increasing slightly to 5.381 ± 0.268 at time point 4, and finally decreasing again at time point 5 to 5.278 ± 0.282.

To get a more clear understanding of the scores observed, growth curve modeling was conducted with SPSS. More specifically, a mixed-effect model was conducted to examine the effects of the time points on the scores. The intercept was significant, b = 5.552, SE = 0.228, t(21.873) = 24.309, p < 0.001. This indicates a strong baseline score. However, the effect of the time point was not significant, b = -0.060, SE = 0.050, t(21.873) = -1.205, p = 0.241. The results also revealed significant variability in repeated measures, Variance = 0.225, SE = 0.042, Wald Z = 5.39, p < 0.001, indicating that scores changed meaningfully across time points. Additionally, there was significant variability in intercepts among participants, Variance = 0.691, SE = 0.282, Wald Z = 2.453, p = 0.014. However, the variance associated with time points was not significant, Variance = 0.023, SE = 0.014, Wald Z = 1.641, p = 0.101.

4.2. RQ2: What is the relationship between the positive experiences of the students and their changes in psychological safety?

This research question was established to explore how the positive experiences of the students, categorized into the 7C's metric, can influence the ebbs and flows of psychological safety in individuals in the population studied here. Since this is a more exploratory investigation, no formal hypothesis was formed. However, it was found in a previous study that communication was the most often mentioned code for freshman engineering students (Cole, O’Connell, Gong, et al., 2022). Therefore, it might have some impact on psychological safety changes. For this part of the analysis, because each response was allowed to be coded into multiple categories, the content analysis resulted in 150 entries for positive experiences.

To examine this impact, quantitative responses from the open-ended responses section of the survey were analyzed. we conducted a Kruskal-Wallis H Test, with the dependent variable being the scores of the change in psychological safety, and the independent variable being the categories of positive interaction. The nonparametric test was selected because, for each of the 7 categories of the independent variable, the number of entries is not equal. This makes the parametric test for the relationship of interest, one-way ANOVA, more subject to errors. Therefore, the Kruskal-Wallis H Test was chosen.

More specifically, the Kruskal-Wallis H test was conducted to determine if there were differences in change in psychological safety scores between the 7 groups of experiences for positive experience: Cohesiveness (n = 14), Communication (n = 32), Composition (n = 9), Conflict (n = 2), Cooperation (n = 41), Coordination (n = 46), Creativity (n = 6). The distribution of change in psychological safety scores was the same for all seven categories, as assessed by visual inspection of a boxplot. The change in psychological safety scores increased from Cohesiveness (Mdn = -0.143) to Communication (Mdn = 0.143), to Composition (Mdn = 0.429), and then it decreased slightly to Conflict (Mdn = 0.071), after which it decreased again to Cooperation (Mdn = -0.143), then it remained the same at Coordination (Mdn = -0.143), and increasing a bit to Creativity (Mdn = 0). However, these differences were not statistically significant, (6) = 7.224, p = 0.301.

4.3. RQ3: What is the relationship between the negative experiences of the students and their changes in psychological safety?

This research question was established to explore how the negative experiences of the students, categorized into the 7C's metric, can influence the building and waning of psychological safety in the population studied here. Since this is a more exploratory investigation, no formal hypothesis was formed. However, it was found in a previous study that communication was the most often mentioned code for freshman engineering students (Cole, O’Connell, Gong, et al., 2022). Therefore, it might have some impact on psychological safety changes. For this part of the analysis, because each response was allowed to be coded into multiple categories, the content analysis resulted in 135 entries for positive experiences.

To examine this impact, quantitative responses from the open-ended responses section of the survey were analyzed. we conducted a Kruskal-Wallis H Test, with the dependent variable being the scores of the change in psychological safety, and the independent variable being the categories of negative interaction. More specifically, the Kruskal-Wallis H Test was conducted to determine if there were differences in change in psychological safety scores between the 7 groups of experiences for positive experience: Cohesiveness (n = 9), Communication (n = 30), Composition (n = 19), Conflict (n = 7), Cooperation (n = 23), Coordination (n = 39), Creativity (n = 8). The distribution of change in psychological safety was the same for all seven categories, as assessed by visual inspection of a boxplot. The change in psychological safety scores increased from Cohesiveness (Mdn = -0.429) to Communication (Mdn = 0.107), then decreased again to Composition (Mdn = -0.286), and decreased further to Conflict (Mdn = -0.429), where it remained the same to Cooperation (Mdn = -0.429), and then increased at Coordination (Mdn = 0.071), and finally increasing more at Creativity (Mdn = 0.143). However, these differences were not statistically significant, χ²(6) = 7.173, p = 0.305.