Abstract

The ASME V&V 10 diagram outlines a framework for the verification and validation of engineering simulations. It comprises an abstraction step to yield a conceptual model that is representative of the reality of interest, and two subsequent branches: a simulation and an experimental branch. Verification activities strictly occur within each branch and are required to ensure that the outcome of each branch is robust, whereas validation requires the quantitative comparison of the outcomes of the two branches. For the simulation branch, the verification activities are entirely within the realm of mathematics only, and the validation by comparison with the experimental outcome is a check on the physics. This diagram is also endorsed by NAFEMS and is included in the joint NAFEMS/ASME document WT09. The V&V 10 diagram provides a robust framework for the verification and validation of the conceptual model, from which both the simulation branch and the experimental branch stem. However, there is no guarantee that this represents robust validation of the reality of interest. The difficulty is that the abstraction process falls outside of the validation comparison between the simulation branch and the experimental branch. If the validation comparison is assumed to be applicable to the reality of interest, this implicitly assumes that the conceptual model has captured all of the relevant physics and is representative of the reality of interest, but this may not be the case. Three CFD case studies are presented which highlight the disconnect between the reality of interest and the conceptual model. For all three case studies, it turns out that thermal radiative transfer can be important, but this may not necessarily be obvious at first and, therefore, this aspect of the physics could be missed during the abstraction process. Indeed, it is even possible that missing the physics of thermal radiative transfer could be commonplace for many other CFD applications, and this may be important for low momentum and buoyancy driven flows. Exploring how good a representation the conceptual model is of the reality of interest is the domain of abstraction validation. However, this is not necessarily straight forward because the abstraction process can include unknown unknowns. A useful recent addition is the Simulation Validation Methods diagram which identifies a range of admissible validation methods under the Engineering Simulation Quality Management Standard (ESQMS), published by NAFEMS in 2020 – see Annex D to the ESQMS. This diagram shows seven admissible forms of validation for the purpose of demonstrating ISO 9001 compliance, and they are shown (from left to right) in order of decreasing strength of validation. Simply by including seven forms of validation, this acknowledges that some forms of validation are better than others, which is in itself useful. Only for the strongest form of validation (1.1 on the left of the diagram) can it be assumed that the conceptual model is actually representative of the reality of interest. For the remaining six forms of validation, there is implicitly an acknowledgement that the conceptual model may representative a departure from the reality of interest. Whilst this may not provide abstraction validation in a quantitative sense, the ESQMS validation diagram provides a standard mechanism to clearly identify shortcomings associated with abstraction validation. It is noted from the case studies presented that the necessary level of validation can depend upon the level of risk associated with the flow application, and the level of dependence upon the simulation predictions. For low risk applications, perhaps one of the weaker forms of validation is appropriate. For high risk applications that are entirely reliant upon the simulation predictions, a stronger form of validation will generally be appropriate.