Hostname: page-component-cb9f654ff-lqqdg Total loading time: 0 Render date: 2025-08-18T12:06:09.766Z Has data issue: false hasContentIssue false
  • English
  • Français

Tailored metrics for assessing the quality of MBSE models

Published online by Cambridge University Press:  16 May 2024

Iris Graessler ,
Dominik Wiechel ,
Deniz Oezcan  and
Patrick Taplick
Iris Graessler
Affiliation:
Heinz Nixdorf Institute, Paderborn University, Germany
Dominik Wiechel*
Affiliation:
Heinz Nixdorf Institute, Paderborn University, Germany
Deniz Oezcan
Affiliation:
Heinz Nixdorf Institute, Paderborn University, Germany
Patrick Taplick
Affiliation:
Behr-Hella Thermocontrol GmbH, Germany
*
dominik.wiechel@hni.upb.de

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

System models are used to merge relevant engineering artefacts and relationships. Therefore, a high model quality must be ensured. Currently, there is no method for defining company-specific metrics to assess system model quality. In a six-step research approach, a method is defined: (1) literature analysis on quality assessment approaches, (2) derivation of success factors, (3) evaluation of approaches, (4) development of a method, (5) application in automotive industry, and (6) evaluation. The method supports system engineers to derive tailored metrics to objectively assess the model quality.

Keywords

model-based systems engineering (MBSE)system modelsmodel qualitydesign methods

Information

Type
Systems Engineering and Design
Information
Proceedings of the Design Society , Volume 4: DESIGN 2024 , May 2024 , pp. 2545 - 2554
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2024.

References

610.12-1990 IEEE Standard Glossary of Software Engineering Terminology, IEEE / Institute of Electrical and Electronics Engineers Incorporated, s.l.Google Scholar
Bansiya, J. and Davis, C.G. (2002), “A hierarchical model for object-oriented design quality assessment”, IEEE Transactions on Software Engineering, Vol. 28 No. 1, pp. 4–17, https://dx.doi.org/10.1109/32.979986.CrossRefGoogle Scholar
Basili, V.R., Briand, L.C. and Melo, W.L. (1996), “A validation of object-oriented design metrics as quality indicators”, IEEE Transactions on Software Engineering, Vol. 22 No. 10, pp. 751–761.CrossRefGoogle Scholar
Basili, V.R., Caldiera, G. and Rombach, H.D. (1994), “The goal question metric approach”, in Encyclopedia of software engineering.Google Scholar
Blessing, L.T.M. and Chakrabarti, A. (2009), DRM, a Design Research Methodology, 1. Auflage, Springer London, Guildford, Surrey, https://dx.doi.org/10.1007/978-1-84882-587-1.CrossRefGoogle Scholar
Briand, L.C., Morasca, S. and Basili, V.R. (1996), “Property-based software engineering measurement”, IEEE Transactions on Software Engineering, Vol. 22 No. 1, pp. 68–86, https://dx.doi.org/10.1109/32.481535.CrossRefGoogle Scholar
Chidamber, S.R. and Kemerer, C.F. (1994), “A metrics suite for object oriented design”, IEEE Transactions on Software Engineering, Vol. 20 No. 6, pp. 476–493, https://dx.doi.org/10.1109/32.295895.CrossRefGoogle Scholar
Walden, David D., Roedler, Garry J. and Forsberg, Kevin (2015), “INCOSE Systems Engineering Handbook Version 4: Updating the Reference for Practitioners”, INCOSE International Symposium, Vol. 25 No. 1.CrossRefGoogle Scholar
Debbabi, M., Hassaïne, F., Jarraya, Y., Soeanu, A. and Alawneh, L. (2010), Verification and validation in systems engineering: Assessing UML/SysML design models, 1. ed., Springer, Berlin, Heidelberg.CrossRefGoogle Scholar
Delligatti, L. (2014), SysML distilled: A brief guide to the systems modeling language, Addison-Wesley, Upper Saddle River, NJ.Google Scholar
DIN (2015), DIN EN ISO 9000: Qualitätsmanagementsysteme - Grundlagen und Begriffe (ISO 9000:2015); No. 9000, 2015th ed., Beuth, Berlin.Google Scholar
Doan, K.-H. and Gogolla, M. (2019), “Quality Improvement for UML and OCL Models Through Bad Smell and Metrics Definition”, in 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Munich, Germany, 15.09.2019 - 20.09.2019, IEEE.Google Scholar
Dromey, R.G. (1995), “A model for software product quality”, IEEE Transactions on Software Engineering, Vol. 21 No. 2, pp. 146–162, https://dx.doi.org/10.1109/32.345830.CrossRefGoogle Scholar
Ernadote, D. (2018), “A Framework for Descriptive Models Quality Assessment”, in 2018 IEEE International Systems Engineering Symposium (ISSE), Rome, 01.10.2018 - 03.10.2018, IEEE, pp. 1–7.Google Scholar
Friedenthal, S. and Burkhart, R. (2015), “Evolving SysML and the System Modeling Environment to Support MBSE”, INSIGHT, Vol. 18 No. 2, pp. 39–41, https://dx.doi.org/10.1002/inst.12020.CrossRefGoogle Scholar
Friedenthal, S., Moore, A. and Steiner, R. (2015), A practical guide to SysML: The systems modeling language, 3. ed., Elsevier; Morgan Kaufmann, Amsterdam, Waltham, Mass.Google Scholar
Giraldo, F.D., España, S., Pastor, Ó. and Giraldo, W.J. (2018), “Considerations about quality in model-driven engineering”, Software Quality Journal, Vol. 26 No. 2, pp. 685–750, https://dx.doi.org/10.1007/s11219-016-9350-6.CrossRefGoogle Scholar
Graessler, I., Wiechel, D. and Oleff, C. (2022), “Extended RFLP for complex technical systems”, in 2022 IEEE International Symposium on Systems Engineering (ISSE), Vienna, Austria, 24.10.2022 - 26.10.2022, IEEE.Google Scholar
Gräßler, I. and Oleff, C. (2022), Systems Engineering: Verstehen und industriell umsetzen, Springer, Berlin, Heidelberg, https://dx.doi.org/10.1007/978-3-662-64517-8.CrossRefGoogle Scholar
Gräßler, I. and Wiechel, D. (2023), “Customized impact analyses for technical engineering changes”, in 2023 18th Annual System of Systems Engineering Conference (SoSe), Lille, France, 14.06.2023 - 16.06.2023, IEEE.Google Scholar
Gräßler, I., Wiechel, D., Koch, A.-S., Sturm, T. and Markfelder, T. (2023), “Methodology for Certification-Compliant Effect-Chain Modeling”, Systems, Vol. 11 No. 3, p. 154, https://dx.doi.org/10.3390/systems11030154.CrossRefGoogle Scholar
Gräßler, I.; Özcan, D.; Preuß, D.: "AI-based extraction of requirements from regulations for automotive engineering DS 125: Proceedings of the 34th Symposium Design for X (DFX2023), 14 2023, S. 163–172.CrossRefGoogle Scholar
Gräßler, I., Wiechel, D. and Pottebaum, J. (2021), “Role model of model-based systems engineering application”, IOP Conference Series: Materials Science and Engineering, Vol. 1097 No. 1, p. 12003.Google Scholar
Hause, M. (2011), ““Are we there yet?” Assessing Quality in Model Based Systems Engineering”, INCOSE International Symposium, Vol. 21 No. 1, pp. 510–522, https://dx.doi.org/10.1002/j.2334-5837.2011.tb01221.x.CrossRefGoogle Scholar
Henderson, K., McDermott, T., van Aken, E. and Salado, A. (2023), “Towards Developing Metrics to Evaluate Digital Engineering”, Systems Engineering, Vol. 26 No. 1, pp. 3–31, https://dx.doi.org/10.1002/sys.21640.CrossRefGoogle Scholar
International Council on Systems Engineering (INCOSE) (2007), INCOSE Systems Engineering Vision 2020, Seattle, WA.Google Scholar
ISO/IEC, Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models No. 25010:2011, ISO.Google Scholar
Kaiser, M., Klier, M. and Heinrich, B. (2007), “How to Measure Data Qualityß - A Metric-Based Approach”, in International Conference on Information Systems (ICIS).Google Scholar
Kilkenny, M.F. and Robinson, K.M. (2018), “Data quality: "Garbage in - garbage out"”, Health information management journal of the Health Information Management Association of Australia, Vol. 47 No. 3.Google Scholar
Krcmar, H. (2015), Informationsmanagement, 6., überarb. Aufl., Springer Gabler, Wiesbaden.CrossRefGoogle Scholar
Lange, C. and Chaudron, M. (2005), “Managing Model Quality in UML-Based Software Development”, in 13th IEEE International Workshop on Software Technology and Engineering Practice (STEP'05), Budapest, Hungary, IEEE, pp. 7–16, https://dx.doi.org/10.1109/STEP.2005.16.CrossRefGoogle Scholar
Lange, C.F., Wijns, M.A. and Chaudron, M.R. (2007), “MetricViewEvolution: UML-based Views for Monitoring Model Evolution and Quality”, in 11th European Conference on Software Maintenance and Reengineering (CSMR'07), IEEE, https://dx.doi.org/10.1109/csmr.2007.32.CrossRefGoogle Scholar
Lehner, D., Vamberszky, S., Wieland, K. and Siegl, D. (2022), “Git-basiertes Qualitätsmonitoring und von Systems Engineering Modellen”, in Koch, W., Wilke, D., Dreiseitel, S. and Kaffenberger, R. (Eds.), Tag des Systems Engineering 2022.Google Scholar
Lindland, O.I., Sindre, G. and Solvberg, A. (1994), “Understanding quality in conceptual modeling”, IEEE Software, Vol. 11 No. 2, pp. 42–49, https://dx.doi.org/10.1109/52.268955.CrossRefGoogle Scholar
Mohagheghi, P. and Aagedal, J. (2007), “Evaluating Quality in Model-Driven Engineering”, in International Workshop on Modeling in Software Engineering (MISE'07: ICSE Workshop 2007), Minneapolis, 2007.Google Scholar
Object Management Group (2004), “OMG: Unified Modeling Language Infrastructure Specification, Version 2.0”, available at: https://www.omg.org/spec/UML/2.0/Superstructure/PDF (accessed 12 October 2020).Google Scholar
Object Management Group (2019), “OMG Systems Modeling Language (OMG SysML™)”, available at: https://sysml.org/.res/docs/specs/OMGSysML-v1.6-19-11-01.pdf.Google Scholar
Parra, E., Alonso, L., Mendieta, R. and La Vara, J.L. de (2019), “Advances in Artefact Quality Analysis for Safety-Critical Systems”, in 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), Berlin, Germany, 27.10.2019 - 30.10.2019, IEEE, pp. 79–84, https://dx.doi.org/10.1109/ISSREW.2019.00047.Google Scholar
Pottebaum, J. and Gräßler, I. (2020), “Informationsqualität in der Produktentwicklung: Modellbasiertes Systems Engineering mit expliziter Berücksichtigung von Unsicherheit”, Konstruktion, Vol. 2020 No. 11-20.Google Scholar
Rohweder, J.P., Kasten, G., Malzahn, D., Piro, A. and Schmid, J. (2008), “Informationsqualität — Definitionen, Dimensionen und Begriffe”, in Hildebrand, K., Gebauer, M., Hinrichs, H. and Mielke, M. (Eds.), Daten- und Informationsqualität, Vieweg+Teubner, Wiesbaden, pp. 25–45, https://dx.doi.org/10.1007/978-3-8348-9266-9_2.CrossRefGoogle Scholar
Rohweder, J.P., Kasten, G., Malzahn, D., Piro, A. and Schmid, J. (2021), “Informationsqualität – Definitionen, Dimensionen und Begriffe”, in Hildebrand, K., Gebauer, M. and Mielke, M. (Eds.), Daten- und Informationsqualität, Springer Fachmedien Wiesbaden, Wiesbaden, pp. 23–43.CrossRefGoogle Scholar
Sargent, R.G. (2013), “Verification and validation of simulation models”, Journal of Simulation, Vol. 7 No. 1, pp. 12–24, https://dx.doi.org/10.1057/jos.2012.20.CrossRefGoogle Scholar
Stvilia, B., Gasser, L., Twidale, M.B. and Smith, L.C. (2007), “A framework for information quality assessment”, Journal of the American Society for Information Science and Technology, Vol. 58 No. 12, pp. 1720–1733.CrossRefGoogle Scholar
Ulrich, H. (1982), “Anwendungsorientierte Wissenschaft”, Die Unternehmung, Vol. 36, pp. 1–10.Google Scholar
United Nations (2021), Regulation - Software update and software update management system No. 156.Google Scholar
VDA QMC Working Group 13 / Automotive SIG (2017), Automotive SPICE: Process Reference Model Process Assessment Model No. Automotive SPICE Version 3.1, 656th ed.Google Scholar
VDI/VDE (2021), VDI 2206 Entwicklung mechatronischer und cyber-physischer Systeme No. 2206, Beuth Verlag GmbH, Düsseldorf.Google Scholar
Wang, R.Y. and Strong, D.M. (1996), “Beyond Accuracy: What Data Quality Means to Data Consumers”, Journal of Management Information Systems, Vol. 12 No. 4, pp. 5–33, https://dx.doi.org/10.1080/07421222.1996.11518099.CrossRefGoogle Scholar
Watts, S., Shankaranarayanan, G. and Even, A. (2009), “Data quality assessment in context: A cognitive perspective”, Decision Support Systems, Vol. 48 No. 1, pp. 202–211, https://dx.doi.org/10.1016/j.dss.2009.07.012.CrossRefGoogle Scholar
Weilkiens, T. (2014), Systems Engineering mit SysML/UML: Modellierung, Analyse, Design, 3rd ed., dpunkt, Heidelberg.Google Scholar
Weiss, D.M. and Basili, V.R. (1985), “Evaluating Software Development by Analysis of Changes: Some Data from the Software Engineering Laboratory”, IEEE Transactions on Software Engineering, SE-11 No. 2.Google Scholar
Weyuker, E.J. (1988), “Evaluating software complexity measures”, IEEE Transactions on Software Engineering, Vol. 14 No. 9, pp. 1357–1365, https://dx.doi.org/10.1109/32.6178.CrossRefGoogle Scholar