Clear Filters
Clear Filters

When running regression tests in Test Manager, how can you tell if all tests have run to completion without skipping any?

12 views (last 30 days)
I am running Simulink Test, running a lot of regression tests in Simulink Test Manager.
I have been using the "stop simulation" workaround to stop the simulation when the Scenario is complete.
What is the best way to make sure that the tests run to completion? Specifically:
  • how can we tell that the test has run but has not got all the way to the end of the scenario? For example, if one of the tests does not pass, the step will not progress to the next transition.
  • is there a way to show the test completes all verify steps and has not missed any?

Answers (1)

Namnendra
Namnendra on 26 Jun 2024 at 10:01
Hello David,
I understand that you want to ensure that tests run to completion and verify that they are executed as expected.
Running regression tests in Simulink Test Manager and ensuring that all tests run to completion can be challenging, especially when dealing with complex scenarios and multiple test sequences. Here are some strategies to help you ensure that tests run to completion and verify that all steps have been executed:
1. Use Test Assessments for Verification
Test Assessments in Simulink provide a way to check conditions and verify that certain criteria are met during simulation. You can use these assessments to ensure that all steps in your test sequence are executed.
Example:
Create a Test Assessment block within your test harness or model, and use `verify` statements to check that each step is completed:
verify(step1Completed, 'Step 1 did not complete');
verify(step2Completed, 'Step 2 did not complete');
verify(step3Completed, 'Step 3 did not complete');
2. Use Custom Criteria in Test Manager
In Simulink Test Manager, you can define custom criteria to check whether the test sequence has completed all steps. This can be done using MATLAB code within the Custom Criteria section.
Example:
% Custom Criteria Code
assert(step1Completed, 'Step 1 did not complete');
assert(step2Completed, 'Step 2 did not complete');
assert(step3Completed, 'Step 3 did not complete');
3. Use Flags to Track Step Completion
You can use flags (boolean variables) to track the completion of each step in your test sequence. These flags can be set within the Test Sequence block and checked at the end of the simulation.
Example:
In your Test Sequence block:
% Step 1
step1Completed = true;
% Step 2
step2Completed = true;
% Step 3
step3Completed = true;
At the end of the simulation, you can check these flags to ensure that all steps have been completed.
4. Use Simulation Data Inspector (SDI)
The Simulation Data Inspector (SDI) can be used to visualize and verify the execution of your test sequence. By logging relevant signals and flags, you can inspect the simulation results to ensure that all steps were executed.
5. Use Model Callbacks
Model callbacks can be used to execute custom MATLAB code at specific points during the simulation. You can use these callbacks to check the status of your test sequence and ensure that all steps are completed.
Example:
In the `StopFcn` callback of your model, add code to check the completion of all steps:
if ~step1Completed || ~step2Completed || ~step3Completed
error('Not all steps were completed.');
end
6. Use Test Sequence Block with Assertion
You can use assertions within the Test Sequence block to ensure that each step is completed before moving to the next one. If an assertion fails, it will stop the simulation and report an error.
Example:
% Step 1
assert(step1Condition, 'Step 1 condition failed');
step1Completed = true;
% Step 2
assert(step2Condition, 'Step 2 condition failed');
step2Completed = true;
% Step 3
assert(step3Condition, 'Step 3 condition failed');
step3Completed = true;
Summary
To ensure that your tests run to completion and verify that all steps are executed:
- Use Test Assessments and `verify` statements.
- Define custom criteria in Test Manager using `assert` statements.
- Use flags to track step completion and check them at the end of the simulation.
- Utilize Simulation Data Inspector (SDI) to visualize execution.
- Implement model callbacks to check step completion.
- Use assertions within the Test Sequence block.
By combining these strategies, you can ensure robust and reliable execution of your regression tests in Simulink Test Manager.
I hope the above steps help you.
Thank you.

Products


Release

R2023b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!