Use Parameters in Class-Based Tests

Often, you need to run a series of tests that are different only in terms of test data. For example, you might want to test that a function produces the expected outputs for different inputs. In this case, the test logic is the same and the only difference between tests is the actual and expected values for each function call. With parameterized testing, you can implement code to iteratively run tests using different data values. When a method is parameterized, the testing framework automatically invokes the method for each parameter value. Therefore, you are not required to implement a separate method for each value.

The testing framework enables you to parameterize your test class at different levels. Additionally, when you call a test class method with multiple parameters, you can specify how the method should be invoked for different combinations of the parameters.

How to Write Parameterized Tests

Classes that derive from the matlab.unittest.TestCase class can implement test parameterization using framework-specific property and method attributes. To provide data as parameters in your class-based test, specify the data using a properties block with an appropriate parameterization attribute. Then, pass the parameterization property as an input argument to one or more methods.

For example, consider the SampleTest class. The class defines a parameterized test because it specifies properties within a properties block with the TestParameter attribute. The properties are passed to Test methods and are used to perform qualifications.

classdef SampleTest < matlab.unittest.TestCase
    properties (TestParameter)
        numericArray = {int16(1),single(zeros(1,4)),magic(3)};
        functionHandle = {@false,@() size([])};
    end
    methods (Test)
        function test1(testCase,numericArray)
            testCase.verifyNotEmpty(numericArray)
        end
        function test2(testCase,functionHandle)
            testCase.verifyWarningFree(functionHandle)
        end
    end
end

The test class results in a parameterized test suite with five elements.

suite = testsuite("SampleTest");
{suite.Name}'

ans =

  5×1 cell array

    {'SampleTest/test1(numericArray=int16_1)'          }
    {'SampleTest/test1(numericArray=1x4_single)'       }
    {'SampleTest/test1(numericArray=3x3_double)'       }
    {'SampleTest/test2(functionHandle=@false)'         }
    {'SampleTest/test2(functionHandle=function_handle)'}

The value assigned to a parameterization property must be either a nonempty cell array or a scalar structure with at least one field. The testing framework uses the property value to specify parameter names and values in the test suite:

• If the property value is a cell array, the framework generates descriptive parameter names from the elements of the cell array by taking into account their values, types, and dimensions.

• If the property value is a structure, the structure fields represent the parameter names and the structure values represent the parameter values. To have full control over parameter names in the suite, define the parameters using a structure instead of a cell array.

How to Initialize Parameterization Properties

When you define a parameterization property, you must initialize the property so that MATLAB^® can generate parameter names and values. You can initialize the property at either test class load time or test suite creation time:

• Class load time: If the property value can be determined at the time MATLAB loads the test class definition, initialize the property using a default value. You can specify the default value in the properties block or using a local function in the classdef file. For more information about assigning values in a class definition, see Evaluation of Expressions in Class Definitions.

  When you initialize a parameterization property at class load time, the parameters associated with the property remain fixed for different test runs. Each time you create a suite from the parameterized test class, the framework uses the same parameter names and values to run the tests. See Create Basic Parameterized Test for an example that uses parameterization properties with default values.

• Suite creation time: If you cannot or do not want to determine the parameters at class load time, initialize the property at suite creation time using a static method with the TestParameterDefinition attribute. When you initialize a parameterization property with a TestParameterDefinition method, the parameters associated with the property can vary for different test runs. Each time you create a suite from the parameterized test class, the framework generates fresh parameter names and values to run the tests. For more information, see Define Parameters at Suite Creation Time.

Once you assign a value to a parameterization property, do not modify it. For example, when you initialize a parameterization property using a default value, you cannot use a TestParameterDefinition method to overwrite the default value.

A parameter might be used by several unit tests. Tests using the same parameter must run independently, without accidentally affecting one another. In addition, a running test must not affect subsequent reruns of the same test. To ensure test run independence, initialize your parameterization properties with value objects. Using handle objects (such as MATLAB graphics objects) as parameter values is not recommended. For more information about the behavior of value and handle objects, see Comparison of Handle and Value Classes.

If you need to test handle objects in your parameterized test, consider constructing them indirectly by using function handles as parameter values and invoking those function handles in tests. For example, write a parameterized test to test the default current point of figures created with the figure and uifigure functions.

classdef FigureTest < matlab.unittest.TestCase
    properties (TestParameter)
        figureType = {@figure,@uifigure};
    end
    methods (Test)
        function defaultCurrentPoint(testCase,figureType)
            fig = figureType();
            testCase.addTeardown(@close,fig)
            cp = fig.CurrentPoint;
            testCase.verifyEqual(cp,[0 0])
        end
    end
end

Specify Parameterization Level

You can parameterize a test class at three levels: class setup, method setup, and test. Parameterizing at each level requires the parameterization properties to have a specific property attribute. For example, at the highest level, a TestClassSetup method can be parameterized using a property defined in a properties block with the ClassSetupParameter attribute. At the lowest level, a Test method can be parameterized using a property defined in a properties block with the TestParameter attribute.

This table shows different parameterization levels and the required method and property attributes for each level.

Parameterization Level	Parameterization Definition		Accessible Parameterization Properties
	Method Attribute	Property Attribute
Class-setup level	TestClassSetup	ClassSetupParameter	ClassSetupParameter
Method-setup level	TestMethodSetup	MethodSetupParameter	MethodSetupParameter and ClassSetupParameter
Test level	Test	TestParameter	TestParameter, MethodSetupParameter, and ClassSetupParameter

A parameterized method can access parameterization properties depending on the level at which the method is defined:

• A parameterized TestClassSetup method can access parameterization properties only with the ClassSetupParameter attribute.

• A parameterized TestMethodSetup method can access parameterization properties only with the MethodSetupParameter or ClassSetupParameter attributes.

• A parameterized Test method can access any parameterization properties.

For an example of how to parameterize a test class at different levels, see Create Advanced Parameterized Test.

Specify How Parameters Are Combined

When you pass more than one parameterization property to a method, you can use the ParameterCombination method attribute to specify how parameters are combined. The testing framework invokes the method for the specified combinations.

This table shows different parameter combination strategies.

ParameterCombination Attribute	Method Invocation
"exhaustive" (default)	Methods are invoked for all combinations of parameter values. The testing framework uses this default combination if you do not specify the ParameterCombination attribute.
"sequential"	Methods are invoked with corresponding values from each parameter. Each parameter must contain the same number of values. For example, if a method is provided with two parameterization properties and each property specifies three parameter values, then the framework invokes the method three times.
"pairwise"	Methods are invoked for every pair of parameter values at least one time. Compared with the "exhaustive" combination, the "pairwise" combination typically results in fewer tests and therefore faster test execution. For example, this test uses the "pairwise" combination to test the size of different matrices. classdef ZerosTest < matlab.unittest.TestCase properties (TestParameter) rowCount = struct("r1",1,"r2",2,"r3",3); columnCount = struct("c1",2,"c2",3,"c3",4); type = {'single','double','uint16'}; end methods (Test,ParameterCombination="pairwise") function testSize(testCase,rowCount,columnCount,type) testCase.verifySize(zeros(rowCount,columnCount,type), ... [rowCount columnCount]) end end end Create a test suite from the class. One possible outcome contains the ten elements listed below. suite = testsuite("ZerosTest"); {suite.Name}' ans = 10×1 cell array {'ZerosTest/testSize(rowCount=r1,columnCount=c1,type=single)'} {'ZerosTest/testSize(rowCount=r1,columnCount=c2,type=double)'} {'ZerosTest/testSize(rowCount=r1,columnCount=c3,type=uint16)'} {'ZerosTest/testSize(rowCount=r2,columnCount=c1,type=double)'} {'ZerosTest/testSize(rowCount=r2,columnCount=c2,type=single)'} {'ZerosTest/testSize(rowCount=r2,columnCount=c3,type=single)'} {'ZerosTest/testSize(rowCount=r3,columnCount=c1,type=uint16)'} {'ZerosTest/testSize(rowCount=r3,columnCount=c2,type=single)'} {'ZerosTest/testSize(rowCount=r3,columnCount=c3,type=double)'} {'ZerosTest/testSize(rowCount=r2,columnCount=c2,type=uint16)'} The testing framework guarantees that the testSize method is invoked for every combination of parameter values specified by any two properties. For instance, for the rowCount and columnCount properties, this table shows the TestSuite array indices corresponding to parameter value combinations. Every combination is represented by at least one Test element. — columnCount Property c1 c2 c3 rowCount Property r1 1 2 3 r2 4 5, 10 6 r3 7 8 9 While the framework guarantees that tests are created for every pair of values at least one time, you should not rely on the suite size, ordering, or specific set of test suite elements.

You can combine parameters at class-setup, method-setup, and test levels. For example, use the two method attributes TestMethodSetup,ParameterCombination="sequential" to specify sequential combination of the method-setup-level parameters defined in the properties block with the MethodSetupParameter attribute.

For examples of how to combine test parameters, see Create Basic Parameterized Test and Create Advanced Parameterized Test.

Use External Parameters in Tests

When you create a parameterized test, you can redefine the parameters by injecting inputs into your class-based test. To provide data that is defined outside of the test file, create a Parameter instance and use the ExternalParameter name-value argument when creating your test suite. For more information, see Use External Parameters in Parameterized Test.