How can i evaluate prediction interval with interval coverage probability (PICP) for probabilistic forecasting?

23 views (last 30 days)
I am trying to use Gaussian Process Regression Models https://au.mathworks.com/help/stats/gaussian-process-regression-models.html
And I would like to use prediction interval assessment: the prediction interval coverage probability (PICP) and the mean prediction interval width (MPIW). PICP measures the reliability of the predictions and shows the percentage of the real values that will be covered by the upper and lower bounds using the attached formula?
Thanks in advance

Answers (1)

Sai Pavan
Sai Pavan on 19 Oct 2023
Hi Israt,
I understand that you are trying to calculate the prediction interval coverage probability (PICP) and the mean prediction interval width (MPIW) for Gaussian Process Regression models.
Please follow the below workflow to calculate PICP and MPIW:
  1. Train your Gaussian Process Regression model using the “fitrgp function
  2. Obtain predictions and prediction intervals using the “predict” function
  3. To calculate PICP:
  • Define the lower and upper bounds of the prediction interval, such as yLower and yUpper, based on the prediction intervals obtained in the previous step.
  • Calculate the number of observations within the prediction interval: numObservations = sum(yLower <= YTest & YTest <= yUpper).
  • Calculate the PICP as the ratio of the number of observations within the prediction interval to the total number of observations: PICP = numObservations / numel(YTest)
4. To calculate MPIW:
  • Calculate the width of each prediction interval: intervalWidths = yUpper - yLower
  • Calculate the mean prediction interval width: MPIW = mean(intervalWidths)
Please refer to the below code snippet to calculate the PICP and MPIW:
gprMdl = fitrgp(XTrain, YTrain); % Train Gaussian Process Regression model
[yPred, ysd, yint] = predict(gprMdl, XTest); % Obtain predictions and prediction intervals
yLower = yint(:, 1);
yUpper = yint(:, 2);
numObservations = sum(yLower <= YTest & YTest <= yUpper);
PICP = numObservations / numel(YTest); % Calculate PICP
intervalWidths = yUpper - yLower;
MPIW = mean(intervalWidths); % Calculate MPIW
Please refer to the below documentations to learn more about “fitrgp” and “predict” functions:
Hope it helps.
Regards,
Sai Pavan

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!