How can I do Validation testing using Test manager?

Hello Sir/Madam,

I am using Test manager to test model in MIL mode for professional project. If I use Test Manager and write test cases then does it come under verification? What should I consider to validate my model in MIL mode If I use Test manager and test harness?

Following are the points I am considering for Validation -

Requirements validation

Coverage testing

Robustness testing with valid, Invalid Inputs

Is Simulink Design Verifier is the way to validate model ?

Need help to clear confusion about verification and validation methods for MIL testing.

Answer 1

[Shishir Dwivedi]

Hello Sunny,

Please find answers to some of your queries below.

Requirements validation

With the term "requirements", I assume you mean “System Requirements”. There can be various aspects to validate system requirements.

To make sure a requirement is validated, you must first create requirement links to establish traceability among different textual requirements, model elements, system/requirements-based test cases etc. This can be done by using Simulink Requirements Toolbox.

Writing system level tests to validate the system requirements is a manual process and depends upon the system scemantics. Simulink Test provides you a platform to author and manage the aforementioned system/requirements based test cases.

Once your test cases are designed/executed in the Simulink Test environment, you can trace tests back to model/textual requirements (with Simulink Requirements) and generate report that include test coverage information from Simulink Coverage.

Please check this video for more information.

Coverage testing

There are two aspects of Coverage Testing,

• Coverage testing on design model: Simulink Design Verifier (SLDV) automatically generates tests from the design model to achieve structural coverage.

• Coverage testing on generated code: Once design model is ready or in the process of rapid prototyping, the user can generate code from the model and get structural coverage on the generated code using SLDV. Additionally, SLDV helps you to get structural coverage on the S-functions used in the model.

Robustness testing with valid, Invalid Inputs

1) Robustness with respect to system requirements: User has to validate system behavior by providing invalid, valid inputs. Which in turn depends upon system requirements/scemantics. (For example, the behavior of the system while providing invalid inputs, like: Reset, latch a fault etc.)

2) Robustness with respect to design model: You can use SLDV to check behavior of design model for invalid inputs. For example,

• Property Proving mechanism: A property can be a simple requirement, such as a signal in your model that must attain a value or range of values during simulation. This feature can be used to test system behavior when signal ranges are falling in invalid equivalence partitions (robust values). This could be a good example to find an invalid property using SLDV property proving analysis.

• Design Error Detection: This feature allows you to identify "hard-to-find" design errors like integer overflow, non-finite and NaN floating-point values, division by zero, and dead logic in the models without requiring extensive tests or simulations. To learn more about Design Error Detection feature, please refer this video.

Is Simulink Design Verifier is the way to validate model?

You can verify a formal requirement (implemented in a model) using Property Proving feature of SLDV A property can be a requirement on the model that involves input and output signals modeled as a logical expression that needs to be proved. I would suggest you to go through this video to learn more on this.

Best Regards,

Shishir.

[Pat Canny]

Shishir covered this very well, and I just want to add a few points:

1. Requirements traceability, which is typically conveyed through links between model and requirements, is one means of validating your requirements. In this specific circumstance, you are ensuring that the requirements are following best practices regarding traceability. Jama Software has a good blog post on the topic, which includes other characteristics of good requirements. Traceability is one tool you can use to ensure your requirements are correct.
2. Collecting coverage at the model level ("model coverage") is another mechanism to validate your requirements. In this circumstance, you would view the coverage with respect to requirements to ensure that your requirements-based test cases fully exercise your design. We recently added features in Simulink Coverage to facilitate this worklow. Here is a link to our Documentation. In most cases, incomplete coverage is due to insufficient testing. However, there may be cases where the incomplete coverage is due to a missing or incomplete requirement, which is a validation exercise.
3. Analysis of your model using Simulink Design Verifier's property proving features can also serve as a validation exercise. In this circumstance, you can check to see whether a requirement can ever be violated (if it is formally defined). Here is a link to an example in our Documentation that goes through a workflow that shows how formal methods can reveal a missing requirement.