polyspace.test.AssessmentResult Class

R2026b

Namespace: polyspace.test

(Python) Review results for a specific assessment in a test

Since R2024b

expand all in page

Description

This Python® class contains test results for a specific assessment in a C/C++ test.

Creation

Syntax

assessmentResult = testCaseResult.AssessmentResults[assessmentIndex]
assessmentResult = testCaseResult.PassedAssessmentResults[assessmentIndex]
assessmentResult = testCaseResult.FailedAssessmentResults[assessmentIndex]

Description

assessmentResult = testCaseResult.AssessmentResults[assessmentIndex] loads the results of assessment number assessmentIndex:

Here, assessmentIndex is the assessment index starting from 0.

example

assessmentResult = testCaseResult.PassedAssessmentResults[assessmentIndex] reads the results of passing assessment number assessmentIndex:

Here, assessmentIndex is the assessment index starting from 0.

example

assessmentResult = testCaseResult.FailedAssessmentResults[assessmentIndex] reads the results of failing assessment number assessmentIndex:

Here, assessmentIndex is the assessment index starting from 0.

example

Properties

expand all

Unique identifier for assessment result, for instance, "ea7222b5-e4fd-453f-ba3b-4c2a7019f9f8".

File containing assessment.

Line number of assessment.

Expression evaluated by assessment, for instance, "lhs == rhs".

Type of assessment specified as an enum class object. The values are:

  • AssessmentType.NONE

  • AssessmentType.VERIFY

  • AssessmentType.ASSERT

  • AssessmentType.ASSUME

Status of assessment, specified as True if the assessment passes or False if the assessment fails.

Expected and actual values when assessment fails, for instance, "lhs="0" rhs="27"".

Custom message when a test failure occurs. You can specify custom messages only for xUnit tests.

Test phase where assessment occurs, specified as one of the following:

  • TestPhase.NONE

  • TestPhase.SUITE

  • TestPhase.SUITE_CONFIG

  • TestPhase.SUITE_SETUP

  • TestPhase.SUITE_TEARDOWN

  • TestPhase.SUITE_TEST_SETUP

  • TestPhase.SUITE_TEST_TEARDOWN

  • TestPhase.TEST

  • TestPhase.TEST_CONFIG

  • TestPhase.TEST_SETUP

  • TestPhase.TEST_TEARDOWN

Since R2026b

Whether this assessment result belongs to a function call sequence assessment, specified as True or False. Use this property to filter assessment results and separate call sequence results from other assessment results.

For more information, see polyspace.project.CallSequenceAssessment.

Examples

collapse all

This example shows how to print the names of failing test cases after running tests in a project.

In general, you generate and manage Polyspace® Test™ results by using classes from the polyspace.project and polyspace.test modules. Before starting, make sure you can import these modules on a Python shell or in a Python script without errors. For more information, see Set Up Python API for Polyspace.

  1. Import the required modules:

    import polyspace.project
import polyspace.test
import os

  2. Add source files and xUnit test files to a project. This example uses some example source and test files available with a Polyspace Test installation. Instead, you can use your own sources and tests.

    examples_path = os.path.join(polyspace.__install_path__, "polyspace", 
                            "examples", "pstest", "Getting_Started_Example")
polyspaceProject = polyspace.project.Project("newProject")
polyspaceProject.Code.Files.add(os.path.join(examples_path, "sources", "utils.c"))
polyspaceProject.IncludePaths.add(os.path.join(examples_path, "includes"))
polyspaceProject.Tests.Files.add(os.path.join(examples_path, "tests", "test.c"))

  3. Run the tests added to the project.

    res = polyspace.test.run(polyspaceProject)

  4. Print the names of failing test cases (along with their suite names) and the file and line numbers of failing assessments in those tests.

    for suiteIndex in range(len(res.TestSuiteResults)):
    # Read result of test suite
    resTestSuite = res.TestSuiteResults[suiteIndex]
    if(resTestSuite.Status.Failed):
        for testIndex in range(len(resTestSuite.TestCaseResults)):
            # Read result of test case
            resTestCase = resTestSuite.TestCaseResults[testIndex]
            if(resTestCase.Status.Failed):
                print(resTestSuite.Name + "/" + resTestCase.Name + "\n")
                for assessmentIndex in range(len(resTestCase.FailedAssessmentResults)):
                    # Read result of failing assessment
                    resAssessment = resTestCase.FailedAssessmentResults[assessmentIndex]
                    print(resAssessment.File + ": Line " + str(resAssessment.Line))

    Alternatively, you can iterate over the test suites, cases and assessments as follows:

    for resTestSuite in res.TestSuiteResults:
    if resTestSuite.Status.Failed:
        for resTestCase in resTestSuite.TestCaseResults:
            if resTestCase.Status.Failed:
                print(f"{resTestSuite.Name}/{resTestCase.Name}\n")
                for resAssessment in resTestCase.FailedAssessmentResults:
                    print(f"{resAssessment.File}: Line {resAssessment.Line}")

After running a test that includes a function call sequence assessment, filter the assessment results to show only those belonging to the call sequence.

Assume you have run tests and obtained results. Use the InCallSequence property to filter assessment results.

res = polyspace.test.run(proj)
stepResult = res.TestSuiteResults[0].TestCaseResults[0].TestStepResults[0]

# Filter only call sequence assessment results
callSeqResults = [a for a in stepResult.AssessmentResults if a.InCallSequence]
for r in callSeqResults:
    print(f"{r.Expression}: {'Passed' if r.Passed else 'Failed'}")

# Filter only non-call-sequence assessment results
otherResults = [a for a in stepResult.AssessmentResults if not a.InCallSequence]
for r in otherResults:
    print(f"{r.Expression}: {'Passed' if r.Passed else 'Failed'}")

Version History

Introduced in R2024b

expand all

See Also

| | |

Topics