Understanding an Engineering Simulation Risk Model

Credibility Theme Strategic Insight 2022

Engineering Simulation is being used more and more broadly to make informed technical and business decisions, especially during the early stages of developing a new product. Use of an Engineering Simulation Risk Model improves credibility through a clearer understanding of the predictive capabilities and “appropriateness” of the simulation(s), thereby increasing confidence in Engineering Simulation influenced decisions.

What is an Engineering Simulation Risk Model?

An Engineering Simulation Risk Model (ESRM) is a Predictive Capability Assessment that includes a set of recommended best practices and associated metrics to understand and manage the “appropriateness” and risk associated with Engineering Simulation influenced decisions.

The approach outlined in this paper for an ESRM is founded on the concept of determining the “appropriateness” of the simulations performed to influence/support the decision(s) being made. This approach to Predictive Capability Assessment is inspired by the NASA-STD-7009A “Use Assessment”, Sandia Predictive Capability Maturity Model, the emerging ASME V&V 40 standard, NAFEMS Simulation Governance and Management Working Group activities, and the ASSESS Credibility Theme discussions.

An ESRM is independent of the type of simulation performed and is intended to account for accuracy, sensitivity, and uncertainty of the simulation along with the criticality and risk associated with the decision(s) being made. The ASSESS initiative has repeatedly stated that, “Every Engineering Simulation that is performed should be done to support either a decision to be made or a decision that has been made.” Therefore, any ESRM must account for the intended use of the simulation being performed.

This ASSESS Strategic Insight Paper starts with a brief review of key efforts in this area at NASA (NASA-STD-7009A) and Sandia (Predictive Capability Maturity Model). Following this, an approach for a generalized ESRM providing a set of recommendations and criteria that could be developed and used in support of Engineering Simulation influenced decisions. The proposed approach includes guidelines, and a potential framework for defining an ESRM. Each organization will need to assess and then tailor the proposed guidelines and framework when implementing an ESRM for their specific applications.

The objective of this ASSESS Strategic Insight Paper is to initiate discussions leading to further development of an effective generalized Engineering Simulation Risk Model that can be used to support the current and dramatically expanding use of Engineering Simulation. Understanding and enabling an effective and consistent ESRM is one key to enabling a significantly broader use of Engineering Simulation in support of more informed business and technical decision making.

The ASSESS Initiative would like to acknowledge the Engineering Simulation Credibility Working Group for its efforts to support advancement of this ASSESS theme and the development of this Strategic Insight Paper along with the efforts of those who participated in Working Sessions at the ASSESS annual Congresses on the Credibility Theme. The ASSESS Initiative would also like to acknowledge the excellent work done in the NASA Standard for Models and Simulations (NASA-STD-7009A) and the Sandia Predictive Capability Maturity Model (PCMM).

The ASSESS Initiative would like to acknowledge specifically the following individuals for their efforts on this ASSESS Strategic Insight Paper.

Primary Author:
Joe Walsh, ASSESS Initiative/intrinSIM

Reviewers:
Rod Dreisbach, Engineering Simulation Consultant
Keith Meintjes, CIMdata
Tina Morrison, US FDA
Martin Steele, NASA
Walter Witkowski, Sandia National Laboratories