Online Training Course

Verification and Validation
in Engineering Simulation

26 - 28 November 2024 | Online (Webex)

ATTENTION:
T​his course is now fully booked! We will schedule another course in spring 2025. To secure your place, please send an e-mail with your contact data to roger.oswald@nafems.org.

Engineering simulation plays an increasing role in industry’s search for competitiveness and technology based innovation at any stage of the design, qualification and certification of their products. Key decisions and product qualification/certification increasingly rely on virtual tests and computational simulations, creating a major paradigm shift in which the objective of physical tests is progressively moving from a demonstration of compliance to a reference for simulation validation. This trend in industry is shown through adoption of new terms such as “realistic simulation” and “virtual testing”. This situation creates new responsibility for the engineer to guarantee the required confidence level.

This new approach requires secured processes for the verification and validation (V&V) of simulations bringing evidence of their predictive capability. In particular, programme managers now require formal evidence on “simulation fit for purpose” on which they can build confidence and take decisions. In addition, the increasing situation for extended enterprise creates new constraints to guarantee safe and robust analysis processes.

At the same time, and due to the economic pressure, V&V activities are frequently seen as an additional cost that can easily be reduced and even fully cut, thus underestimating the induced risks. In addition, V&V is not easy to implement because of the diversity of involved persons: managers, simulation experts, test specialists, software developers and quality controllers, software vendors…

Course objectives and benefits

This Master Class is especially dedicated to V&V and credibility assurance methodologies for numerical simulation in Engineering. Participants will:

• Develop their knowledge in V&V including fundamental concepts, most recent methodologies and contents of existing standards.
• Understand essential links between the product validation/certification and simulation V&V.
• Learn how to plan and prioritize simulation V&V including physical tests programmes.
• Understand validation test issues and improve synergy between virtual and physical tests in the context of validation.
• Learn how to build business cases allowing for justification of V&V plans.
• Understand simulation V&V organisation and management issues and best practises
• Learn how to implement reporting to bring visibility and confidence to all managers concerned with simulation outcomes.
• Be able to adapt or tailor the course methodologies to their specific industrial context, and further improve their V&V processes and plans.

The course has been taught over 39 times since 2013 all over Europe.

Who should attend?

• Primarily experienced engineers and senior analysts in charge of simulation activities or preparing to take new responsibility in the management of simulation, especially regarding V&V responsibility
• Managers in charge of engineering simulation teams and willing to improve their simulation V&V knowledge
• Programme/project managers and stakeholders from regulatory bodies who need to make critical decisions based on engineering simulation results and wish to increase their understanding of simulation V&V.

Participants should have a solid experience in engineering simulation for the design and development of industrial products.

Course context

Are concerned with this course all industry sectors confronted with numerical simulation V&V, and primarily those sectors where simulation is playing a critical role for the product design or certification. The course is established on the solid knowledge of the tutors and the vision they have developed from their rich industrial experience, mainly within the aerospace and nuclear energy sectors and is primarily illustrated by FEA/Structure Analysis examples. The course is therefore of greatest interest to engineers involved in this technical domain but will also appeal to engineers from other simulation domains because most of the V&V methodologies which are presented are generic in nature.

The course is neutral and independent of any particular software solution.

We limit the number of participants to a small group to facilitate dialog and exchanges between participants.

2,5 day Master Class Program

Introduction

• Introduction, scope of the course
• Simulation fundamentals, simulation process
• Simulation credibility assurance for decision making, simulation criticality

Verification & Validation of industrial products

• Basic concepts : ISO V&V concepts, qualification & certification of industrial products
• Systems Engineering : main concepts, focus on system V&V
• Hierarchical validation
• Impact of product innovation

Simulation management with focus on credibility

• Simulation management versus governance
• Management of simulation capabilities

• Key management processes for simulation credibility: Simulation Process and Data Management, competence management

Simulation V&V foundations & standards

• Basic V&V concepts for Engineering Simulation: Verification, Validation, Uncertainty Quantification, predictive capability, examples

• V&V processes and responsibilities

• Guides and Standards
  A short history of V&V standardization
  Main guides and standards : AIAA CFD Guide, ASME V&V 10 & 20, NASA STD 7009A, ASME V&V 40
  Introduction on credibility assessment

Code verification

• Introduction on Simulation codes (CSM, CFD, CEM…)
• Software Quality Assurance – Metrics, SQA versus V&V
• Method of exact solutions
• Method of manufactured solutions
• Benchmarks, other methods
• Examples: codes, methods, benchmarks
• Check-list, matrix of verification

Calculation verification

• Metrics
• Model verification- Basic tests, run tests
• Error estimation with focus on spatial discretization, Richardson extrapolation, GCI, specific error estimators (finite elements, finite volumes…)
• Mesh refinement: uniform or adaptive?
• Singularities
• Time discretization errors
• Examples

Validation and simulation / test synergy

• Overall validation process &planning.
• Validation core process, simulation & test collaboration
• Differences between calibration and validation
• Validation uncertainties: experimental uncertainties: ASME V&V 10.1 example
• Accuracy assessment & validation metrics :
  metrics for scalar quantities: deterministic, statistical (area metric, ASME V&V 20, error metric...), metrics for waveforms, ASME V&V 10.1 example

• Validation trends: validation and model acceptance (coverage analysis...), practical validation (different levels of validation rigor, use of historical data from similarity...).

Simulation Uncertainty Quantification (UQ)

• Introduction on HPC
• Reduction of models
• Framework vs objectives
• UQ methods: Monte-Carlo and derivatives, FORM-SORM
• Other methods: LHS, P-Box, response surface…
• Sensitivity analysis: ANOVA
• UQ software, examples
• Summary on best practices
• UQ and optimization

V&V Implementation strategies

• V&V implementation and management issues
• Simulation/V&V and risk informed decision making: criticality assessment, risk informed credibility assurance
• Simulation/V&V benefits & costs, ROI
• Recommended practises
  PIRT analysis for V&V planning
  Credibility assessment, reporting to the decision maker: survey of credibility assessment procedures, focus on NASA STD 7009A (CAS) and PCMM
  Organization and management