risk.validation.proportionOfFailuresTest

R2026b

Proportion of failures test

Since R2025b

Syntax

hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials)

hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials,ConfidenceLevel=confidenceLevel)

[hPOFTest,POFOutput] = risk.validation.proportionOfFailuresTest(___)

Description

hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials) returns the proportion of failures (POF) test result or results, hPOFTest, for a given set of probabilities, events, and trials. The output is 1 if the test rejects the null hypothesis at the 95% confidence level, or 0 otherwise. Probability contains numeric values that represent quantities such as a value-at-risk (VaR) level for a portfolio.

hPOFTest = risk.validation.proportionOfFailuresTest(Probability,NumEvents,NumTrials,ConfidenceLevel=confidenceLevel) specifies the confidence level for the POF test.

[hPOFTest,POFOutput] = risk.validation.proportionOfFailuresTest(___) also returns a structure POFOutput that contains a table of summary metrics. Specify POFOutput as the second output argument with any of the input argument combinations in the previous syntaxes.

Examples

Perform Proportion of Failures Test

Perform a proportion of failures (POF) test to determine whether 5% or 1% is an accurate probability for an event occurring in a single trial, given a sample of 250 trials in which three events occurred.

[hPOFTest,POFOutput] = risk.validation.proportionOfFailuresTest([0.05 0.01],3,250)

hPOFTest = 2×1

     1
     0

POFOutput = struct with fields:
    Results: [2×8 table]

POFOutput.Results

ans = 2×8 table
    RejectPOFTest     PValue      LogRatio    CriticalValue    ConfidenceLevel    NumEvents    NumTrials    Probability
    _____________    _________    ________    _____________    _______________    _________    _________    ___________

          1          0.0010083     10.812        3.8415             0.95              3           250          0.05    
          0            0.75799    0.09494        3.8415             0.95              3           250          0.01    

The output includes the hypothesis test results and their corresponding p-values. The test results indicate that enough evidence exists to reject the null hypothesis for the 5%, but not the 1% probability. This conclusion is supported by the small p-value for the test statistics corresponding to 5%.

Input Arguments

Probability — Probability values

numeric scalar between 0 and 1 | numeric vector of values between 0 and 1

Probability values, specified as a numeric scalar between 0 and 1, or a vector of values between 0 and 1. For VaR models, Probability can be a set of VaR levels for a portfolio or portfolios.

When Probability is a vector, risk.validation.proportionOfFailuresTest performs a test for each of its elements. If one or both of NumEvents and NumTrials are also vectors, they must be of the same length as Probability.

Example: 0.95

NumEvents — Number of events observed

nonnegative integer | vector of nonnegative integers

Number of events observed, specified as a nonnegative integer or vector of nonnegative integers. For VaR models, NumEvents contains the number of VaR exceptions.

When NumEvents is a vector, risk.validation.proportionOfFailuresTest performs a test for each of its elements. If one or both of Probability and NumTrials are also vectors, they must be of the same length as NumEvents.

Example: 30

NumTrials — Number of trials

positive integer | vector of positive integers

Number of trials, specified as a positive integer or vector of positive integers. For VaR models, NumTrials contains the number of time periods in the backtesting window.

When NumTrials is a vector, risk.validation.proportionOfFailuresTest performs a test for each of its elements. If one or both of Probability and NumEvents are also vectors, they must be of the same length as NumTrials.

Example: 250

confidenceLevel — Confidence level

0.95 (default) | numeric scalar in the range (0,1)

Confidence level of the hypothesis test, specified as a numeric scalar in the range (0,1).

Output Arguments

hPOFTest — Hypothesis test results

numeric vector containing values of 1 or 0

Hypothesis test results, returned as a numeric vector containing values of 1 or 0.

• A value of 1 rejects the null hypothesis at the specified confidence level.

• A value of 0 fails to reject the null hypothesis at the specified confidence level.

POFOutput — Output metrics

structure

Output metrics, returned as a structure with a single field, Results. Results contains a table with the following columns:

• RejectPOFTest — Numeric value indicating whether the null hypothesis was rejected. This field represents the same value as hPOFTest.

• PValue — Numeric value in the range [0,1] representing the p-value for the hypothesis test. A small value indicates that the null hypothesis might not be valid.

• LogRatio — Numeric value representing the test statistic for the hypothesis test.

• CriticalValue — Numeric value representing the critical value for the hypothesis tests.

• ConfidenceLevel — Numeric value representing the confidence level for the hypothesis tests.

• NumEvents — Numeric value representing the number of events.

• NumTrials — Numeric value representing the number of trials.

• Probability — Numeric value representing the probability of an event.

For more information about the POF test and its corresponding statistics, see More About.

More About

Proportion of Failures Test

Kupiec's proportion of failures (POF) test [1] investigates the accuracy of an event's probability, given a number of events and observations. The null hypothesis is that the probability is accurate.

The POF test statistic is

$$LRatioPOF=-2\log \frac{{\left(1-pVaR\right)}^{N-x}pVa{R}^x}{{\left(1-\frac{x}{N}\right)}^{N-x}{\left(\frac{x}{N}\right)}^x}=-2\left[\left(N-x\right)\log \frac{N\left(1-pVaR\right)}{N-x}+x\log \frac{N\left(pVaR\right)}{x}\right],$$

where N is the number of trials, x is the number of events, and pVar = 1 − VaRLevel. By the properties of the logarithm,

$$LRatioPOF=-2N\log \left(1-pVaR\right)\text{, if }x=0\text{,}\text{and}LRatioPOF=-2N\log pVaR\text{, if }x=N\text{.}$$

This test statistic is asymptotically distributed as a chi-square distribution with 1 degree of freedom. The corresponding p-value is

$$PValuePOF=1-F\left(LRatioPOF\right)\text{,}$$

where F is the cumulative distribution function of a chi-square variable with 1 degree of freedom.

For more information about the POF test, see [1].

Alternative Functionality

You can use the varbacktest object function pof to perform a proportion of failures test on a timeseries of portfolio outcomes. The pof function allows you to perform proportion of failures tests on multiple timeseries with different VaR confidence levels.

References

[1] Kupiec, P. "Techniques for Verifying the Accuracy of Risk Management Models." Journal of Derivatives. Vol. 3, 1995, pp. 73–84.

Version History

Introduced in R2025b