1. Introduction
Although there are studies suggesting the benefits of using external stimuli in idea generation (Reference Vasconcelos and CrillyVasconcelos & Crilly, 2016), such stimuli can also impair the creativity process and lead designers down fixed thought paths (Reference Badke-Schaub, Gonçalves and CardosoBadke-Schaub et al., 2011). This phenomenon is referred to as design fixation (DF) (Reference Jansson and SmithJansson & Smith, 1991). DF is an unintentional attachment to certain key indicators, e.g. features or principles of existing concepts or objects and their subsequent reuse in a different context without analysing the appropriateness for their use (Reference Cardoso and Badke-SchaubCardoso & Badke-Schaub, 2011). The phenomenon occurs when a designer sees an example of an existing product and then develops a new product with similar features, with one possible factor for DF being the use of prototypes (Reference YoumansYoumans, 2011).
Studies of the effect of prototyping on DF discuss the relationship inconsistently. While some studies highlight the positive effect of prototyping by emphasizing the usefulness of physical prototypes in design work (Reference YoumansYoumans, 2011), other studies point to negative effects such as sunk cost effects and limited design space exploration (DSE) due to the time needed to build said prototypes in the first place (Reference Viswanathan and LinseyViswanathan & Linsey, 2013). Most of these studies have in common that they are conducted in an experimental setting, addressing synthetic design tasks and covering only short sequences of the design process (e.g. an ideation session). While such study designs seem well-suited for many research questions, the contradicting prototyping-related findings motivate the assumption that such study designs may not be appropriate for analysing the effect of prototyping on DSE behaviour, i.e. analysing the effect of downstream activities or decisions on the creative process.
DSE behaviour describes patterns of design synthesis and analysis along the divergent and convergent phases of the ideation process (Design Council UK, 2005) including the outcome of the individual activities. Usually, the outcome of the creative process is measured e.g. in terms of the number of ideas and novelty of ideas. Reference Paravizo and CrillyParavizo & Crilly (2024) provide an overview of alternative metrics for analysing the outcome, which goes far beyond this. Their work also supports our assumption that study designs must be refined for properly studying DSE behaviour.
Based on a literature review of idea generation and creativity studies, this paper presents a refined study design and initial findings from a pilot study in which the study design was tested.
The aim of future research goes beyond the investigation of typical patterns in DSE and aims to provide methodological support for mitigating undesirable effects on ideation. The impact of prototyping on design space exploration behaviour will be investigated. Furthermore, it should also be possible to analyse the effect of different interventions on design fixation.
This leads to the following research question regarding the presented study of this paper: (RQ) “What modifications could further improve the pilot study design?”.
This paper will shortly recap the literature, with a focus on a critical analysis of the reported study designs and their suitability for the research outlined above and will also provide an overview of key indicators for identifying DF during DSE. Furthermore, a revised study design is proposed before it is tested in a pilot study. The results and insights are presented and discussed and recommendations for improving future studies are made before the paper concludes with a summary and an outlook for future research.
2. Design fixation in design space exploration
In product development, the term Design Fixation (DF) was coined by Reference Jansson and SmithJansson & Smith (1991). Analysing the results of four experiments they observed that product developers orientate themselves significantly on existing concepts when generating new product ideas. Based on their research, many studies have been conducted to study the effects of DF in varying conditions and settings (Reference Vasconcelos and CrillyVasconcelos and Crilly, 2016). To steer the currently scattered fixation research in a common direction, standardise the term and develop a structured methodology, an international congress of researchers specified DF with a working definition: “Design fixation is a state in which someone engaged in a design task undertakes a restricted exploration of the design space due to an unconscious bias resulting from prior experiences, knowledge or assumptions.” (Reference Crilly and CardosoCrilly & Cardoso, 2017).
2.1. Key indicators for design fixation
Existing literature on DF has identified several fixation factors which can be used to measure DF, and where possible key indicators for analysis can be derived deductively. Therefore, this paper refers to them as “deductive key indicators” from now onwards. In contrast, “inductive key indicators” for DF are generated through data processing, for example from transcripts of design workshop observations (for more details see section 4.2).
In the following paragraph deductive key indicators derived from the literature and relevant for the presented study will be introduced. One of the original ideas of DF research relates to participants’ fixation on the task and sample solutions provided by the task (Reference Jansson and SmithJansson & Smith, 1991). Another possibility is fixation in the designs based on previous personal experiences (Reference CrillyCrilly, 2015). Fixation may also be categorised into three groups: unconscious adherence to the influence of previous designs and conscious fixation, whereby the latter can be further divided into the conscious blocks to change or an intentional resistance to new ideas (Reference Youmans and ArciszewskiYoumans & Arciszewski, 2014). When confronted with a decision, designers also usually only consider one or at best a few alternatives. As soon as one alternative exceeds a certain threshold and appears acceptable, a decision is often made in favour of that alternative (Reference Stempfle and Badke-SchaubStempfle & Badke-Schaub, 2002). Fixating on just one idea and ignoring other ideas therefore also counts towards a possible key indicator (Reference Jia, Jiang, Hu and QiJia et al., 2023). Depending on the background of the test participants, fixations on certain methods might occur. These can relate to both processes and procedures (Reference Brennan, Miney, Simpson, Jablokow and McCombBrennan et al., 2023; Reference CrillyCrilly, 2015). Sunk costs are another potential fixation indicator that occurs more frequently as resource utilisation progresses (Reference Viswanathan and LinseyViswanathan & Linsey, 2013). Recent studies show that fixation rates also depend on the source of the idea. Self-generated ideas are more likely to trigger fixation than ideas from others. In addition, familiar products and examples also lead to higher fixations than unfamiliar products or abstract examples (Reference Jia, Jiang, Hu and QiJia et al., 2023). Finally, cognitive biases like ownership bias, availability bias or the bandwagon effect can lead to fixation effects (Reference Fillingim, Shapiro and FuFillingim et al., 2023).
To counteract these fixation indicators, the DF literature suggests mitigation strategies in various degrees of complexity and depth, including non-experimental research like surveys, interviews or logbooks to give further insights into the design processes of designers or changes in the provided examples. These changes might be from different disciplines, new and unknown examples, or changing the timing of the introduction (Reference Vasconcelos and CrillyVasconcelos & Crilly, 2016). Giving the designers a logical context - like a re-engineering task - brings good opportunities to introduce examples naturally (Reference Crilly and CardosoCrilly & Cardoso, 2017). There is also work regarding fixation differences between groups and individuals (Reference Blosch-Paidosh and SheaBlosch-Paidosh & Shea, 2021; Reference YoumansYoumans, 2011). Furthermore, participating in workshops to broaden the view and knowledge of the designers can lead to eliminating the fixation effects of both individuals and groups (Reference Brennan, Miney, Simpson, Jablokow and McCombBrennan et al., 2023). Lastly, another mitigation of fixation might be the conscious and extensive use of prototypes in iterations and parallel concept testing as well as the use of virtual prototypes (Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and WoodCamburn et al., 2015).
2.2. Shortcomings of current DF research
In 2017, Crilly & Cardoso observed that DF research was characterized by five main points: It was distinctive, productive, disconnected from other research disciplines, uncritical of its procedures and undirected (Reference Crilly and CardosoCrilly & Cardoso, 2017). The last three points will be looked at in more detail. While more recent studies on DF show greater diversity (Reference Blosch-Paidosh and SheaBlosch-Paidosh & Shea, 2021; Reference Böhmer, Hostettler, Richter, Lindemann, Conradt and KnollBöhmer et al., 2017; Reference Brennan, Miney, Simpson, Jablokow and McCombBrennan et al., 2023; Reference Calpin and MenoldCalpin & Menold, 2023; Reference Das and YangDas & Yang, 2022), most experiments (Reference Vasconcelos and CrillyVasconcelos & Crilly, 2016) on DF mainly deal with the idea generation phase of the development process only and are based on Reference Jansson and SmithJansson & Smith (1991) experiments. Accordingly, most studies involve participants working on designs to develop multiple solutions to simple problems due to time constraints, typically under an hour. Participants are divided into groups, with some exposed to external stimuli before or during problem-solving. The effects of these stimuli are measured objectively (e.g., number of solutions) or subjectively (e.g., innovation level). The results are then analysed by comparing group performance.
For this reason, one major drawback is the similarity of the study designs, and it can be argued in some cases that a fixation in the general DF research is evident. DF is often presented as an unfavourable phenomenon, and most studies show ways to avoid, mitigate or overcome it (Reference Vasconcelos and CrillyVasconcelos & Crilly, 2016). The shortcomings of individual experiments are thereby not sufficiently mitigated by the application of other studies. (Reference Vasconcelos and CrillyVasconcelos & Crilly, 2016).
Another point of criticism is the gap between the way creative activities take place “in the lab” and “in the wild” (Reference CrillyCrilly, 2019). Study situations might differ significantly from real-world scenarios, making it difficult to compare these cases and draw valid conclusions. While studies are addressing a longer period than just the ideation workshop (Reference Böhmer, Hostettler, Richter, Lindemann, Conradt and KnollBöhmer et al., 2017; Reference Das and YangDas & Yang, 2022; Reference Zheng, Ritter and MillerZheng et al., 2018), no evident sources are aiming to observe the whole duration of the natural development of a design project with the need to submit many paper-prototypes (sketches) and a physical prototype at the end, which might translate to real-world tasks and projects where a finished product is expected at the end. To overcome this limitation in DSE observation duration, a refined study design is proposed below which depicts both the longitudinal elements of design processes as well as their influence on the ideation workshop behaviour.
3. Product Design and Development course (PDD)
The pilot study was set in the master’s course “Product Design and Development”(PDD) of the Otto von Guericke University (OVGU) Magdeburg. By integrating the pilot study into the course, the participants could be observed for a longer period than just the ideation workshop, therefore negating some of the previously mentioned issues regarding time restrictions.
PDD is part of an optional course lineup, where students can freely choose and enrol into a set number of different courses to gain credits for their master’s program. PDD is an introductory-level engineering design class, where, in addition to the lectures in the first weeks of the course, groups of participants were engaged in a longitudinal design task following the phases of the product development process. The course aimed to create an innovative design concept for a given mechanical device in an ideation workshop, with the need for the prototypical realisation of the concept as a full-scale paper model at the end of the semester as well as a visual representation of the entire design process in the form of a digital design portfolio. The prototype and the design portfolio were graded. Lack of participation in the task led to deductions in the grade to incentivise proactivity. The researcher was both involved in the course as a lecturer and in the workshop as a moderator.
3.1. Course schedule
The first four weeks of the course were used to teach basic knowledge for product development including design methodology, embodiment design and behaviour guidelines for design workshops. This was followed by a product development task over twelve weeks. This phase started with the presentation of the design task and an ideation workshop (WS) to create an initial design idea for further development during the semester. Five weeks after the workshop, the design progress of each group was presented to the class during a mandatory milestone presentation. At this stage, the aim was to have a finished design concept and present a roadmap for the implementation of the physical prototype. Following the milestone presentation, participants had seven weeks to finish the physical prototype and a digital design portfolio of the product. This included the documentation of the whole design process from the first sketches to the realisation of the physical prototype as well as the digital enhancement and rendering of the prototype as a finished product. Designing the template for the portfolio was part of the process as well. The students additionally had the opportunity to contact the lecturer for further feedback on the progression of the design task over weekly meetings which were offered throughout the whole semester. The course schedule is shown schematically in the upper half of Figure 1.
3.2. Ideation workshop
The workshop duration was 90 minutes and was mainly self-moderated by the participants. The researcher only intervened as a moderator if the workshop rules were not followed to ensure minimal bias in terms of methods and ideas generated. Participation in the workshop was obligatory. The task was to generate ideas for a modern, innovative Bluetooth Speaker Box (BTB). After an initial 15-minute design task briefing conducted by the researcher, the groups started a self-moderated 45-minute idea-generation session. This is where the intervention, in the form of different ideation methods, took place (see section 4.3). Afterwards, in a 30-minute, open discussion, the participants presented their ideas to each other and chose one as their initial design concept to elaborate on during the semester. The participants were not informed about the research topic to reduce the risk of biases towards DF as this was meant as a baseline study to further iterate upon.
3.3. Participants
In the course eight participants enrolled from two engineering master’s programs, “Sports and Technology” (three participants) and “Integrated Design Engineering” (five participants) of the OVGU Magdeburg. The group consists of an equal number of men and women. The participants were assigned into two groups of four members each, trying to represent all genders and study programs equally. While every participant had engineering qualifications, only one of them claimed to have previous experience with design tasks.
4. Study design
To tackle the research question “What modifications could further improve the pilot study design?”, and to investigate the impact of prototyping on DSE behaviour a refined study design is tested.
The working hypothesis for the pilot study states that the need to realise a product as a physical prototype and the associated longitudinal effects influence the DSE behaviour of designers at the beginning of their design process, potentially leading to DF effects. Taking into account the shortcomings of the mostly punctual studies from the literature review, a longitudinal observation is additionally implemented in the workshop observation. The ideation workshop is also the starting point for various interventions which should enable the investigation of further factors for DF, for example, interventions in the form of different ideation methods.
To analyse the data for DSE behaviour and detect possible DF during the design process, two kinds of key indicators were used. Deductive key indicators are taken from existing literature (see section 2.1), focussing on general fixation behaviour patterns like cognitive biases (Reference Fillingim, Shapiro and FuFillingim et al., 2023), taking on an overarching, holistic role to evaluate the behaviour of the group dynamics.
Inductive key indicators on the other hand are local fixation points in the product design process, where participants were drawn to during their workshop, generated from the analysis of the transcript of the recordings. Inductive key indicators therefore relate to the product properties, e.g. form, material or certain features. The inductive key indicators were also used to track longitudinal fixation behaviour patterns in their ideas by comparing their presence/absence in the following milestone and final submission.
4.1. Study schedule and data collection
First, the WS was recorded, resulting in 180 min (2 × 90 min) of video and audio material. The data was then analysed (see section 4.2) and used to generate inductive key indicators for DF.
Mandatory feedback questionnaires were submitted weekly by the groups. They were roughly based on further study designs (Reference Das and YangDas & Yang, 2022; Reference Fillingim, Shapiro and FuFillingim et al., 2023; Reference Loweth, Daly, Hortop, Strehl and SienkoLoweth et al., 2021) and were used to track the development of the design task and to look for potential DF patterns during the design process.
In addition to the final prototype, the complete documentation of the design process also served as material to check the design process for DF. The course schedule can be seen schematically in Figure 1.
Figure 1. Course and pilot study schedule
Data was collected according to the ethics standards of the Scientific Integrity Service Centre guidelines of the OVGU university, fulfilling both the DFG guidelines and the legally binding statutes of the OVGU.
4.2. Determine and analyse inductive key indicators for design fixation
The recordings of the workshops were manually transcribed and analysed using MAXQDA (VERBI Software, 2024). A mixed method approach including qualitative content analysis (Mayring, 2000) and grounded theory (Strübing, 2019) was used to generate inductive key indicators from the transcript as a measurement of participants’ fixation on certain ideas or features.
Transcribed segments were identified as an inductive key indicator for DF when they revolved around ideas or product features repeatedly and were then coded accordingly. The codes are named after said idea or feature. Coding was realised as soon as the idea/feature was mentioned in the discussion either literally or analogously after its initial introduction to the group. Content-related passages were coded as a single occurrence.
Both the final product as well as the product design presented during the milestone were compared to the initial ideas of the groups and analysed for the presence of the inductive key indicators. The transcript of the workshop was also analysed for the deductive indicators (see section 2.1). Additionally, the completed weekly questionnaires were checked for both the inductive and deductive key indicators.
4.3. Interventions to detect design fixation
For the exploration of how different interventions affect DF and therefore DSE, different ideation methods varying in collaboration style were implemented. One ideation method solely focused on individual work, and the other included an exchange of ideas (group work) midway through the ideation, Group 1 (G1 - individual work) used an adaptation of the brainwriting method where a pool of ideas is generated by writing them down. In contrast to brainstorming, the brainwriting method involves working individually and without verbal exchange (Reference VanGundyVanGundy, 1984). The adaptation allows participants to also use sketches to underline their ideas. Brainwriting is carried out in three successive phases (Reference Pahl, Beitz, Feldhusen and GrotePahl et al., 2007). The individual phases, numbered 1-3, consist of twelve minutes of idea generation and a three-minute interruption, based on the Pomodoro technique by Reference CirilloCirillo (2018). For the whole duration of the ideation method, the participants were instructed to not talk about their ideas.
The gallery method is used by group 2 (G2 - group work) and combines individual and group work. It is also divided into three phases. Firstly, individual ideas are generated using sketches and verbal explanations in ideation phase 1 (twelve minutes). In the subsequent association phase (fifteen minutes), the group members can comment on the ideas of all individuals and initiate further design options by negating and recombining the ideas. In the final phase, ideation phase 2 (twelve minutes), the participants can incorporate these new ideas into their designs (Reference Pahl, Beitz, Feldhusen and GrotePahl et al., 2007).
5. Results
Both groups submitted a final digital design portfolio of their BTB. The designs did not change significantly over the twelve weeks. Figure 2 shows three stages of the design process, from the initial concept of the ideation workshop to the milestone presentation and the final digital renderings of the product. The physical prototype was also submitted but had no relevance to the research and therefore will not be further examined. G1 designed a modern gramophone, while G2 settled on portable speakers which could be integrated into a lighting system at home.
Figure 2. Project results over the semester
5.2. Workshop observations and generated ideas
17 ideas were presented by G1, 15 of which fall under the broad category of holistic ideas (e.g. surround sound speaker, gramophone) and two discussing possible features (solar panels and external battery). Additionally, a great discrepancy could be observed in the total number of ideas generated per participant, ranging from three to seven.
The perceived creativity of each idea depended heavily on the individual abilities of the participants. During the open discussion phase, most of the ideas were considered equally. A convergence of the design space could be observed in the second half of the discussion phase, with two options available until the final decision was made at the end of the ideation workshop. There was no indication of quantitative factors used by the group to choose the final idea for further development.
G2 generated a smaller number of ideas which were discussed in more detail and depth compared to G1. Out of the 14 ideas presented in the association phase, four were identified as broad ideas, while the others focused on basic shapes, functions or features. Only the first phase of the gallery method was used for generating novel ideas (divergent DSE behaviour), whereas the other phases were used for detailing and combining those ideas into concepts (convergent DSE behaviour). Anecdotal comments from the participants during their idea workshops additionally mentioned the physical prototype as one factor to restrict their ideas.
The dynamic of the open discussion can be represented by the speaking time for each participant (see Figure 3). It is calculated by dividing the participants’ active speaking time by the total duration of the transcribed discussion (twenty minutes). In comparison, participant 2 of G1 (38,3%) as well as participant 1 (36,5%) and 4 (30,2%) of G2 have dominant speaking times. The individual graphs of G2 are shorter in length, indicating shorter speaking times and higher rates of disruption.
Figure 3. Overview of speaking times during the open discussion after the ideation methods
5.2. Key indicators for design fixation
As described in section 4.2, the transcript was analysed and eleven indicators for G1 and twelve for G2 were identified. Table 1 shows these inductive key indicators and their relative coding frequency (rF). rF is calculated automatically by the analysing software by dividing the coded characters for one indicator by the total sum of characters coded as an inductive key indicator. During analysis, some segments were coded with more than one indicator, therefore the rF might exceed 100% when added up.
Table 1. Inductively generated key indicators of both groups and their relative coding frequencies.
Table 2 shows the percentage of fixation for both groups calculated automatically by the analysing software using the ratio of all characters coded as key indicators to the total number of characters in the transcript. Fixation was analysed both concerning the inductive key indicators shown in Table 1 and with the deductive key indicators mentioned in section 2.1. A higher number of code segments was considered for G2, as the association phase of the workshop was included in the determination of the inductive factors in addition to the open discussion (thirty minutes). G2, with 23% for the inductive factors and 20% for the deductive indicators, has a higher fixation rate than G1 with 15% for the inductive and 16% for the deductive indicators. In addition, G2 has a higher inductively indicated fixation than deductively, whereas it is the other way around for G1.
Table 2. Percentages of fixation indicators of both groups.
Every inductive key indicator present in the initial design can be identified at both the milestone presentation as well as the final renderings of the product in the digital design portfolio (see Figure 2).
Unfortunately, the weekly feedback questionnaires representing the longitudinal aspect of the pilot study could not be analysed sufficiently neither for the indicators nor for design trends by the groups due to a lack of qualitatively adequate answers by the participants.
6. Discussion
6.1. Design space exploration
G1 explored the design space over the whole course of the ideation phase, which led to a higher number of ideas compared to G2. Since no interruption and no discussion were intended in this ideation method, the whole duration can be argued to be a divergent design space exploration phase.
The three sets of ideation phases proved to be difficult for the participants since no input was given from the outside and communication between participants was prohibited. Therefore, they solely relied on their creativity for the idea generation. This led to a slowdown of the ideation process due to a profound lack of ideas and examples to orientate, especially in the latter half of the ideation method. Additionally, feedback given afterwards suggested that the given timeframe was too long for unguided ideation: a lack of concentration or motivation could be observed. Strategically placed examples or motivation might be needed to guide the open ideation phases. This could be in the form of options to reach for examples, which are then shared among the group, or time-triggered examples.
The brainwriting method eliminates the influence of other members on the individual ideas, but therefore lacks critical feedback as well, making the method’s DSE capabilities highly dependent on the individual skills of the participants and their ability to critically evaluate themselves.
There was no clear structure given by the moderator regarding the workshop to gain insights into the group dynamics without influences from the outside. G1 was unable to structure or moderate confidently during open discussions. They also lacked initiative, which led to a slowdown, and, in some instances, discussions therefore came to a halt. A moderator therefore will be beneficial to motivate all group members to join the discussion.
Due to the mutual feedback, G2 had no issues creating ideas during the second ideation phase, with some suggesting even longer ideation phases to be able to implement the feedback. G2 only diverged into the design space in the first part of the gallery method. Afterwards, they started to converge due to conclusions drawn from the association phase which affected DSE behaviour significantly. The exchange led to the dismission of multiple ideas, either due to lack of presentation or due to certain individuals pushing their ideas too much.
As mentioned in section 5.2, G2 set their own moderation rules which they wanted to adhere to independently from the workshop briefing. Controversially, those self-set rules were not followed at all during the discussions, Prominent individuals continued to interfere during the introduction and discussion of ideas. The moderator interfered multiple times to reinforce the workshop rules
Additionally, a high bias towards own ideas could be observed from some participants. Combined with their high percentage of speaking time (see Figure 3, participant 1 and 4 (G2)), this might have led the group to choose particular ideas over others. It can be hypothesized that individual members therefore might have had a greater influence on the group dynamics than the intervention, which needs to be regulated in the following studies.
Due to the mentioned problems, it cannot be concluded with certainty whether the restricted exploration of the design space was due to the used ideation method being of a restrictive nature or if the team structure was interfering with the intervention.
6.2. Influences of embedding the pilot study in an optional class
As mentioned in section 3, the class was optional, which resulted in a clash of schedules and the difficulty of organising weekly meetings since there were two different master’s programs involved. This restricted the time to work on the prototype. The lack of engagement could be mitigated in future studies by more mandatory deadlines and reports, especially in the longitudinal prototyping part.
It was observed that the need to create a physical paper prototype limited the participants’ generation of ideas (divergence). Both groups opted for rather simple and, above all, implementable solutions. The grading at the end of the course may have reinforced this. Furthermore, there was not enough incentive to explore more complex, risky or unconventional ideas.
Weekly questionnaires were supposed to track hints of apparent DF and restricted DSE behaviour in the created prototypes. While both groups submitted the weekly questionnaires the gained insights were insignificant due to the lack of quality of the answers.
6.3. Possible study design improvements
Based on the insights from section 5, possible improvements for the study design will be discussed. The first insight was the higher idea quality reached with the intervention method utilizing group work. While G1 is spanning a wider design space, G2 is developing a deeper level of maturity per idea due to the early involvement of the other group members. But more precise metrics for the DSE behaviour would have to be established here to make the result more meaningful - Reference Paravizo and CrillyParavizo & Crilly (2024), for example, are already making a few suggestions. On the other hand, the presence of dominant individuals in G2 also decreased the number of possible ideas. This might have played a more crucial role in this study’s outcome than the interventions. To mitigate this effect, an individual pretest battery should be implemented deliberately to check for team compatibility before forming the groups.
Furthermore, a moderator should be able to mitigate some of the effects caused by the loose structure of the workshop by guiding the participants and directing the focus to relevant topics. A longer, guided ideation workshop without the need to choose one design at the end of the workshop and therefore no artificial restriction through the given design task should also help in spanning a wider design space at the beginning of the design process.
The weekly questionnaire should be simplified and provide more detailed instructions to reduce the barrier for students to fill out the questionnaire in a qualitative manner. This should help get a better understanding from the observation of the design process for the related conclusions that can be drawn from this information regarding the DSE behaviour during the workshop and the longitudinal part of the study. A crucial point for improvement would be to disconnect the researcher from an active role in the course moderation and lecture. Therefore, a separate project facilitator is needed to enable the researcher to act as an observer.
7. Conclusion
In principle, the study design is suitable for assessing DF. The results show the possibilities of examining the DSE behaviour of different groups, with differences already visible in the chosen interventions for the pilot study. Therefore, it can be argued that a refined study design will be able to capture changes in DSE during the design process. It was shown that group work increases the quality of the results. However, metrics for the description of the design space would have to be defined in detail to measure them more precisely.
An effect was observed that the prototyping obligation establishes a mindset regarding the realisability of ideas - both groups decided on rather simple and, above all, buildable solutions (section 5.2). However, in the current evaluation, this cannot yet be mapped by measurable factors. The influence here needs to be analysed in more detail. When analysing the factors, more attention should be paid to implementation tips or thoughts regarding production. It would also be important to analyse what role the comments play in the evolution of an idea.
Overall, this supports the hypothesis that the obligation to implement the result, in this case, the prototypes, leads to a change in behaviour. For future study designs there is the possibility that, longitudinal sections and their influence on DF could thus be simulated, which could help to remedy the weaknesses identified in research about the consideration of only short periods (e.g. idea workshop) in study designs. The requirement to realise a prototype could also make it possible to simulate a real use case. In addition, several weaknesses were discussed, and remedies were proposed (see section 6.3).
Albeit there were problems regarding the longitudinal part of the study design, an early convergence could be observed in both groups. Due to dominant participants and the additional association phase, the trend of early convergence could be observed earlier in G2.
Future work involves further investigations into the actual influence of prototyping as well as adjusting the measuring metrics accordingly to evaluate the created ideas of the participants regarding DSE.
Open access
