How to determine the subset of a test suite which needs to be rerun due to changes to a project's codebase?

Hi,

I'm working with a large codebase which has a number of test suites for different modules within it.

We're using a CI pipeline to run a suite of tests. However, the codebase is getting large enough that the runtime of these tests is quite long. Most of the tests are operating on code that hasn't been changed and so shouldn't need to be re-executed.

What's the best way to determine the tests which do need to be rerun? I've tried using calls to listImpactedFiles based on the files listed by git diff, but this call seems to take a long time itself.

Additionally, once I've found the tests which need to be executed, is there a good way to collate the results of these tests with the existing results such that Test Manager can produce a report containing the full suite?

Apologies if this is a repeated question - I've tried searching answers for details on this as I'm sure it's a reasonably common issue to run into, but I wasn't able to find anything.

Thanks.

Answer 1

[Guilherme Costa Nascimento]

I recommend you take a look at the CI/CD Automation support package: https://www.mathworks.com/products/ci-cd-automation.html

One of its main capabilities is performing incremental builds, which only run tasks with outdated results to reduce your build time. This is especially helpful if you are commiting your models and code frequently so that the changed artifacts are only a fraction of the total test suite.

On your second question, to generate a single set of results for the different test runs, all you need to do is load all of the results onto the test manager. If they are all loaded you'll be able to save a unified report.

If you are collating the results from multiple pipeline runs, you'll need to export the results as a .mldatx file at the end of each run and store them somewhere so that you can import them later onto the test manager to generate the unified report.