The usage of micromobility devices is growing to promote sustainable transportation, prompting manufacturers and regulators to enable its safe integration into urban environments. This has created the need for a tool to evaluate such devices. This paper presents the development of a versatile dynamometer design for verification and validation of micromobility devices by emulating real-world conditions while capturing vehicle and battery performance in real-time. A custom Graphical User Interface (GUI) is used to control and configure the system, as well as enabling the user to analyze and save incoming data. Six devices were chosen from distinct categories to collect data and demonstrate the capabilities and modularity of the dynamometer. The results reflect the ability of the dynamometer to be used for standardized testing of various micromobility devices.

Virtual reality (VR) based product evaluation is a growing area of research, but has not yet been studied in the context of design education. In this study, participants evaluated four pairs of product design student models in physical and VR form using a custom VR application. The models included a variety of product types and a variety of prototyping materials. Participants rated the physical and VR models using a rubric adapted from a study of design student prototyping. Significant differences were found in VR versus physical ratings of some model evaluation categories, but only for certain products. The majority of participants preferred the physical model evaluation over the VR evaluation. Our findings suggest that VR product evaluation may be suitable for use in design education contexts, especially when budget or time for physical prototyping is limited.

Creativity is a fundamental aspect of design that can bring us novel and useful products. However, measuring creativity in design can always be challenging as there is a lack of standardized quantification methods and the inherent limitations of mathematical modelling. Previous approaches often rely on human experts to assess design creativity. Still, humans can be subjective and biased in their evaluation procedures. Recent advancements in AI have inspired us to integrate LLMs as evaluators in engineering design. In this study, we utilize LLMs to assess the novelty and usefulness of design ideas. We developed an evaluation procedure and tested it using design samples. Experimental results demonstrate that the proposed method enhances creativity evaluation capabilities across various LLMs and improves the alignment between LLM and human expert assessments.

Most innovation performance measurement approaches focus on ex-post outcome data, leaving decision-makers without timely guidance during the early phases of new product development (NPD). This gap is particularly critical in high-risk, high-regulation industries such as Urban Air Mobility (UAM), where long development cycles, regulatory hurdles, and uncertain user adoption demand real-time, in-process innovation metrics. In this paper, we propose a Desirability-Feasibility-Viability (DFV) framework that links key innovation phases (Discovery, Development, and Commercialization) to leading indicators that track innovation progress before market entry. Using UAM as an illustrative case study, we demonstrate how our framework enables stakeholders to navigate uncertainty, optimize resource allocation, and make data-driven innovation decisions.

Design method validation is fundamental to ensure that design methods achieve their objectives in the intended situations and are accepted in practice. Although various method validation approaches have been developed, there is still a lack of practical guidance for planning validation studies based on project characteristics. To address this, an intensity map of the validation effort is presented as the core of a scenario-based planning approach. It categorizes projects according to the novelty of the method and the state of research on the problem or the research area, enabling the required validation studies, their sequence and validation criteria to be identified. Thereby, researchers can plan validation studies and estimate the required effort situation-based, allowing for a better alignment with their individual project characteristics before starting studies.

This novel contributions reveal how environmental regulations drive engineering design costs, focusing on the emblematic case of packaging. Using a regulatory database and simulation-based modeling, we evaluate functional expansion as a key driver of cost escalation, identifying its volume effect (rising costs from added environmental functions) and scope effect (increased interdependencies among ecosystem actors). The findings offer a simulated cost envelope to support engineering design teams in their forecasts, but also underscore the hurdles of sustainably managing these regulatory-driven costs in the packaging product system, by benchmarking cost trajectories against sustainability metrics, such as carbon pricing.

This paper explores the multifaceted concept of design theories value, challenging traditional views of science and philosophy and proposing a novel framework for evaluation. Through critical analysis, considering design theories like C-K theory, PSI, GDT, and CDP, and insight from the history of science, we establish the need for a new value model of design theories that includes design-related and other general properties such as generativity, robustness, and impact on practice. We adapt a recently developed system value model (SVM) to consider the diverse perspectives of design theory stakeholders. Our framework is tested on the PSI theory, demonstrating its applicability. This paper redefines how we perceive and measure the value of design theories, offering insights that could influence future research and practice in design science.

This study explores the design of a compatibility evaluation framework for integrating 3PL warehouse clients into semi-automated warehouse setups. Using Action Design Research (ADR), an artifact was developed that combines data-driven decision-making (DDD) and multi-criteria decision analysis. The framework, implemented in Microsoft Power BI, enables the evaluation of client compatibility based on configurable criteria and relevant metrics. It was co-created with stakeholders and tested using data from 33 warehouse clients, demonstrating its practical value in identifying operational fit while facilitating data-driven discussions. The study highlights the potential of structured decision frameworks in environments with limited data, offering generalizable insights for 3PL warehouses and similar contexts.

The Consensual Assessment Technique (CAT) is one of the most effective and commonly used design evaluation methods. However, it fails to capture implicit cognitive processes and has mainly been studied in a homogenous design modality. To bridge this gap, the present study investigates the impact of design ideas represented in different modalities (i.e., text-only, sketch-only, text + sketch) on design evaluations for creativity, novelty, and usefulness, and examine human gaze patterns during the evaluation process. Our findings showed that novice raters exhibit higher interrater reliability and greater convergence in visual attention when rating ideas containing sketches compared to text-only design modality, highlighting the value of visual elements in design evaluations.

Large Language Models offer a novel approach with low barriers to entry to potentially improve knowledge transfer in product development. After identifying knowledge barriers from literature that are potentially addressable through LLM-based applications, we analyze two GDPR-compliant LLM applications - ChatGPT Enterprise and Langdock - examining their key features: assistants and chatbots for both, and prompt libraries and LLM-based file search for Langdock. Then, we evaluate each feature’s potential to mitigate each barrier. Our findings show that assistants and chatbots provide wide-ranging support across many barriers, whereas prompt libraries and file search deliver targeted solutions for a narrower set of specific challenges. Given the numerous influencing factors and the rapidly evolving field of LLMs, the study concludes with a research agenda to validate the theoretical findings.

The objective of this research is to identify and synthesize metrics to assess virtual prototypes in product design. The metrics are identified from literature and practitioners (novice/experienced designers and design faculty members), and evaluation categories are constituted. The identified metrics and constituted evaluation categories from: (a) literature and practitioners, and (b) across various practitioner groups, are compared. 144 and 29 distinct metrics are identified from literature and practitioners, resulting in 15 and 9 evaluations categories, respectively. The metrics from the practitioners is a subset of the metrics from the literature. The differences between: (a) literature and practitioners, and (b) across various practitioner groups, suggest the need for support to help practitioners choose relevant metrics for their prototyping context from an encompassing list.

With the increase of service robots, understanding how people perceive their human-likeness and capabilities in use contexts is crucial. Advancements in generative AI offer the potential to create realistic, dynamic video representations of robots in motion. This study introduces an AI-assisted workflow for creating video representations of robots for evaluation studies. As a comparative study, it explores the effect of AI-generated videos on people's perceptions of robot designs in three service contexts. Nine video clips depicting robots in motion were created and presented in an online survey. Videos increased human-likeness perceptions for supermarket robots but had the same effect on restaurant and delivery robots as images. Perceptions of capabilities showed negligible differences between media types. No significant differences in the effectiveness of communication were found.

Estimating consumer impressions of a product’s appearance is essential. However, this is not easy because of the variety in consumers’ tastes and differences in how consumers and designers experience design. Multimodal foundation models trained on datasets from the internet could be applicable for the estimation; however, it remains unclear if the models’ tastes are similar to those of consumers or experts like designers. Therefore, we conducted surveys in which consumers and designers rated the appearance of car wheels. In addition, a foundation model estimated the visual impression of the wheels. The model’s ratings were more similar to those provided by designers than consumers. Therefore, the models could have tastes similar to those of experts because the datasets could contain advertisements and reviews written by experts or product owners who have opinions on product appearance.

To investigate human behavior in spatial environments, researchers commonly implement video-based behavioral analysis, which is time-consuming and tedious. With improvements in algorithmic performance and expansions in behavior datasets, vision-based AI demonstrates great potential to support human behavior analysis and understanding in design research automatically. To bridge this gap, we proposed a framework for utilizing vision-based AI models for spatial behavior analysis tasks in design research and utilize it in applications. This work offers new insights for design researchers, pointing toward strategies for refning AI-enhanced human behavior analysis and integrating emerging AI technologies into the study of human behavior in design settings.

Controllable summarization models are typically limited only to a short text, such as a topic mention, a keyword, or an entity, to control the output summary. At the same time, existing models for controllable summarization are prone to generate artificial content, resulting in unreliable summaries. In this work, we propose a method for controllable abstractive summarization that can exploit arbitrary textual context from a short text to a collection of documents to direct the focus of the generated summary. The proposed method incorporates a sentence BERT model to extract an embedding-based representation of the given context, which is then used to tag the most representative words of the input document towards this context. In addition, we propose an unsupervised metric to evaluate the faithfulness of the topic-oriented sentences of the generated summaries with respect to the input document. Experimental results under different zero-shot setups demonstrate that the proposed method surpasses both state-of-the-art large language models (LLMs) and controllable summarization methods. The generated summaries are both reliable and relevant with respect to the input document.

Epistemic democrats indirectly evaluate democratic decisions by directly evaluating the inputs into the election. However, the fundamental problem of measurement in the philosophy of science shows that procedures are often as difficult to evaluate as outcomes. This paper brings this highly refined framework into political philosophy to show that epistemic democrats face an analogous ‘fundamental problem of evaluation’. This cross-fertilization of political philosophy with the philosophy of science shows that the quality of democratic mechanisms and their inputs regarding their ability to track the truths of justice is as difficult to evaluate as the quality of the resulting decisions themselves.

This paper describes the evaluation of a simple service adaptation and associated brief training for NHS Talking Therapies for Anxiety and Depression (NHS TTad) staff on working with autistic people. A simple question regarding whether clients identified themselves as autistic and an associated data system flag was introduced to an NHS TTad service. A brief training regarding the use of the flag, a brief overview of autism and a consideration of general adaptations that might help autistic people was developed. Core outcomes of confidence and therapy self-efficacy were reported for pre-training, immediately post-training and at three months post-training. At three-month follow-up, six therapists were interviewed to explore changes in practice following the training. There were significant changes in confidence and therapeutic self-efficacy post-training that were maintained at three-month follow-up. Therapists report several changes to practice that they related to the training. This is the first paper to describe and evaluate training for therapists in NHS TTad on working with autistic people.

Key learning aims

1. (1) To describe some of the challenges to NHS TTad services in working with autistic people.

2. (2) To describe the system adaptation and therapist training introduced to this service and the approach to evaluation.

3. (3) To report outcomes from the evaluation of the training for NHS TTad therapists in working with autistic adults.

4. (4) To consider further research and practice in the processes to make NHS TTad services more accessible and effective for autistic adults.