Test Performance Using Scripts or Functions

This example shows how to create and run a script-based or function-based performance test that times the preallocation of a vector using four different approaches.

Write Performance Test

Create a performance test in a file named preallocationTest.m in your current folder. In this example, you can choose to use either the following script-based test or the function-based test. The output in this example is for the function-based test. If you use the script-based test, then your test names will be different.

Script-Based Performance TestFunction-Based Performance Test
vectorSize = 1e7;

%% Ones Function
x = ones(1,vectorSize);

%% Indexing With Variable
id = 1:vectorSize;
x(id) = 1;

%% Indexing On LHS
x(1:vectorSize) = 1;

%% For Loop
for i=1:vectorSize
    x(i) = 1;
end
 
function tests = preallocationTest
tests = functiontests(localfunctions);
end

function testOnes(testCase)
vectorSize = getSize();
x = ones(1,vectorSize());
end

function testIndexingWithVariable(testCase)
vectorSize = getSize();
id = 1:vectorSize;
x(id) = 1;
end

function testIndexingOnLHS(testCase)
vectorSize = getSize();
x(1:vectorSize) = 1;
end

function testForLoop(testCase)
vectorSize = getSize();
for i=1:vectorSize
    x(i) = 1;
end
end

function vectorSize = getSize()
vectorSize = 1e7;
end

Run Performance Test

Run the performance test using the runperf function.

results = runperf("preallocationTest.m")
Running preallocationTest
.......... .......... .......... .......... .......
Done preallocationTest
__________


results = 

  1×4 TimeResult array with properties:

    Name
    Valid
    Samples
    TestActivity

Totals:
   4 Valid, 0 Invalid.
   8.7168 seconds testing time.

The results variable is a 1-by-4 TimeResult array. Each element in the array corresponds to one of the tests defined in preallocationTest.m.

Display Test Results

Display the measurement results for the second test. Your results might vary.

results(2)
ans = 

  TimeResult with properties:

            Name: 'preallocationTest/testIndexingWithVariable'
           Valid: 1
         Samples: [4×7 table]
    TestActivity: [9×12 table]

Totals:
   1 Valid, 0 Invalid.
   0.87973 seconds testing time.

As indicated by the size of the TestActivity property, the performance testing framework collected nine measurements. This number of measurements includes five measurements to warm up the code. The Samples property excludes warm-up measurements.

Display the sample measurements for the second test.

results(2).Samples
ans =

  4×7 table

                       Name                       MeasuredTime         Timestamp             Host        Platform                 Version                             RunIdentifier            
    __________________________________________    ____________    ____________________    ___________    ________    __________________________________    ____________________________________

    preallocationTest/testIndexingWithVariable      0.096513      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1
    preallocationTest/testIndexingWithVariable      0.097008      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1
    preallocationTest/testIndexingWithVariable      0.096777      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1
    preallocationTest/testIndexingWithVariable      0.097157      14-Oct-2022 14:04:00    MY-HOSTNAME     win64      9.14.0.2081372 (R2023a) Prerelease    7ff84b09-8a58-4961-bfcb-75c86f4a3ae1

Compute Statistics for Single Test Element

Display the mean measured time for the second test. To exclude data collected in the warm-up runs, use the values in the Samples property.

sampleTimes = results(2).Samples.MeasuredTime;
meanTest2 = mean(sampleTimes)
meanTest2 =

    0.0969

Compute Statistics for All Test Elements

To compare the different preallocation methods, create a table of summary statistics from results. In this example, the ones function was the fastest way to initialize the vector to ones. The performance testing framework made four measurement runs for this test.

T = sampleSummary(results)
T =

  4×7 table

                       Name                       SampleSize      Mean      StandardDeviation      Min        Median       Max   
    __________________________________________    __________    ________    _________________    ________    ________    ________

    preallocationTest/testOnes                         4        0.016716       0.00018455        0.016515    0.016699    0.016952
    preallocationTest/testIndexingWithVariable         4        0.096864        0.0002817        0.096513    0.096892    0.097157
    preallocationTest/testIndexingOnLHS               15        0.024099        0.0025168        0.022721      0.0232    0.031685
    preallocationTest/testForLoop                      4         0.79044         0.016054         0.78203     0.78261     0.81452

Change Statistical Objectives and Rerun Tests

Change the statistical objectives defined by the runperf function by constructing and running a time experiment. Construct a time experiment that collects two warm-up measurements and runs the test a variable number of times to reach a sample mean with a 4% relative margin of error within a 98% confidence level.

Create a test suite.

suite = testsuite("preallocationTest");

Construct a time experiment with the specified requirements, and run the test suite.

import matlab.perftest.TimeExperiment
experiment = TimeExperiment.limitingSamplingError("NumWarmups",2, ...
    "RelativeMarginOfError",0.04,"ConfidenceLevel",0.98);
resultsTE = run(experiment,suite);

Running preallocationTest
.......... .......... .......... ..........
Done preallocationTest
__________

Compute the summary statistics for all the test elements.

T1 = sampleSummary(resultsTE)
T1 =

  4×7 table

                       Name                       SampleSize      Mean      StandardDeviation      Min        Median       Max   
    __________________________________________    __________    ________    _________________    ________    ________    ________

    preallocationTest/testOnes                        16        0.017424         0.001223        0.016911    0.017056    0.021938
    preallocationTest/testIndexingWithVariable         8        0.099153        0.0039523        0.096619    0.097218     0.10736
    preallocationTest/testIndexingOnLHS                4        0.022985       0.00018664        0.022843    0.022921    0.023257
    preallocationTest/testForLoop                      4         0.80613         0.005993         0.79979     0.80591      0.8129

See Also

| | |

Topics