trackErrorMetrics

Track error and NEES

Description

The `trackErrorMetrics` System object™ provides quantitative comparisons between tracks and known truth trajectories.

To generate track assignment metrics:

1. Create the `trackErrorMetrics` object and set its properties.

2. Call the object with arguments, as if it were a function.

Creation

Syntax

``errorMetrics = trackErrorMetrics``
``errorMetrics = trackErrorMetrics(Name,Value)``

Description

````errorMetrics = trackErrorMetrics` creates a `trackErrorMetrics` System object with default property values.```

example

````errorMetrics = trackErrorMetrics(Name,Value)` sets properties for the `trackErrorMetrics` object using one or more name-value pairs. For example, ```metrics = trackErrorMetrics('MotionModel','constvel')``` creates a `trackErrorMetrics` object with a constant velocity motion model. Enclose property names in quotes.```

Properties

expand all

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the `release` function unlocks them.

If a property is tunable, you can change its value at any time.

Error function format specified as `'built-in'` or `'custom'`.

• `'built-in'` – Enable the `MotionModel` property.

This property is a convenient interface when using tracks reported by any built-in multi-object tracker, and truths reported by the `platformPoses` object function of a `trackingScenario` object. The default estimation error function assumes `tracks` and `truths` are arrays of structures or arrays of objects.

• `'custom'` – Enable custom properties: `EstimationErrorLabels`, `EstimationErrorFcn`, `TruthIdentifierFcn`, and `TrackIdentifierFcns`. These properties can be used to construct error functions for arbitrary tracks and truths input arrays.

Desired platform motion model, specified as `'constvel'`, `'constacc'`, `'constturn'`, or `'singer'`. This property selects the motion model used by the `tracks` input.

The motion models expect the `'State'` field of the `tracks` to have a column vector containing these values:

• `'constvel'` — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

• `'constacc'` — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

• `'constturn'` — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

• `'singer'` — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The `'StateCovariance'` field of the `tracks` input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the `'State'` field of the tracks input.

Labels for outputs of error estimation function, specified as an array of strings or cell array of character vectors. The number of labels must correspond to the number of outputs of the error estimation function. Specify the error estimation functions using the `EstimationErrorFcn` property.

Example: `{'posMSE','velMSE'}`

Dependencies

To enable this property, set the `ErrorFunctionFormat` property to `'custom'`.

Data Types: `char` | `string`

Error estimation function, specified as a function handle. The function determines estimation errors of truths to tracks.

The error estimation function can have multiple scalar outputs and must have the following syntax.

` [out1,out2, ...,outN] = estimationerror(onetrack,onetruth)`
The number of outputs must match the number of entries in the labels array specified in the `EstimationErrorLabels` property.

`onetrack` is an element of the `tracks` array passed in as input `trackErrorMetric` at object updates. `onetruth` is an element of the `truths` array passed in at object updates. The `trackErrorMetrics` object averages each output arithmetically when reporting across tracks or truths.

Example: `@errorFunction`

Dependencies

To enable this property, set the `ErrorFunctionFormat` property to `'custom'`.

Data Types: `function_handle`

Track identifier function, specified as a function handle. Specifies the track identifiers for the `tracks` input at object update. The track identifiers are unique string or numeric values.

The track identifier function must have the following syntax:

` trackID = trackIDentifier(tracks)`

`tracks` is the same as the `tracks` array passed as input for `trackErrorMetric` at object update. `trackID` is the same size as `tracks`. The default identification function handle, `@defaultTrackIdentifier`, assumes `tracks` is an array of structures or objects with a `'TrackID'` field name or property.

Dependencies

To enable this property, set the `ErrorFunctionFormat` property to `'custom'`.

Data Types: `function_handle`

Truth identifier function, specified as a function handle. Specifies the truth identifiers for the `truths` input at object update. The truth identifiers are unique string or numeric values.

The truth identifier function must have the following syntax:

` truthID = truthIDentifier(truths)`

`truths` is the same as the `truths` array passed as input for `trackErrorMetric` updates. `truthID` must have the same size as `truths`. The default identification function handle, `@defaultTruthIdentifier`, assumes `truths` is an array or structures or objects with a `'PlatformID'` field name or property.

Dependencies

To enable this property, set the `ErrorFunctionFormat` property to `'custom'`.

Data Types: `function_handle`

Usage

To estimate errors, call the track error metrics object with arguments, as if it were a function (described here).

Syntax

``[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)``
``[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)``
``[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)``
``[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs)``

Description

example

````[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)` returns the metrics `posRMSE` – Position root mean squared error`velRMSE` – Velocity root mean squared error`posANEES` – Position average normalized-estimation error squared`velANEES` – Velocity average normalized-estimation error squared for constant velocity motion at the current time step. `trackIDs` is the set of track identifiers for all tracks. `truthIDs` is the set of truth identifiers. `tracks` are the set of tracks, and `truths` are the set of truths. `trackIDs` and `truthIDs` are each a vector whose corresponding elements match the track and truth identifiers found in `tracks` and `truths`, respectively.The RMSE and ANEES values for different states are calculated by averaging the errors of all tracks at the current time step. For example, the position RMSE value, `posRMSE`, is defined as: $\text{posRMSE}=\sqrt{\frac{1}{M}\sum _{i=1}^{M}‖\Delta {p}_{i}{‖}^{2}}$where M is the total number of tracks with associated truth trajectories in the current time step, and $\Delta {p}_{i}={p}_{track,i}-{p}_{truth,i}$is the position difference between the position of track i, ptrack,i, and the position of the corresponding truth, ptruth,i, at the current time step. The RMSE values for other states (`vel`, `pos`, `acc`, and `yawRate`) are defined similarly. The position ANEES value, `posANEES`, is defined as: $\text{posANEES}=\frac{1}{M}\sum _{i=1}^{M}\Delta {p}_{i}{}^{T}{C}_{p,i}^{-1}\Delta {p}_{i}$where Cp,i is the covariance matrix corresponding to the position of track i at the current time step. The ANEES values for other states (`vel`, `pos`, `acc`, and `yawRate`) are defined similarly.To enable this syntax, set the `ErrorFunctionFormat` property to `'built-in'` and the `MotionModel` property to `'constvel'`.```
````[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)` also returns the metrics `accRMS` – Acceleration root mean squared error`accANEES` – acceleration average normalized-estimation error squared for constant acceleration motion at the current time step.To enable this syntax, set the `ErrorFunctionFormat` property to `'built-in'` and the `MotionModel` property to `'constacc'`.```
````[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)` also returns the metrics `yawRateRMSE` – yaw rate root mean squared error`yawRateANEES` – yaw rate average normalized-estimation error squared for constant turn-rate motion at the current time step.To enable this syntax, set the `ErrorFunctionFormat` property to `'built-in'` and the `MotionModel` property to `'constturn'`.```
````[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs)` returns the user-defined metrics `out1`, `out2`, ... , `outN`.To enable this syntax, set the `ErrorFunctionFormat` property to `'custom'`. The number of outputs corresponds to the number of elements listed in the `EstimationErrorLabels` property, and must match the number of outputs in the `EstimationErrorFcn`. The results of the estimation errors are averaged arithmetically over all track-to-truth assignments.```

Tip

These usage syntaxes only calculate the RMSE and ANEES values of all tracks with associated truths at the current time step. To obtain the cumulative RMSE and ANEES values for each track and truth, use the `cumulativeTrackMetrics` and `cumulativeTruthMetrics` object functions, respectively. To obtain the current RMSE and ANEES values for each track and truth, use the `currentTrackMetrics` and `currentTruthMetrics` object functions, respectively.

Input Arguments

expand all

Track information, specified as an array of structures or objects. For built-in trackers such as `trackerGNN` or `trackerTOMHT`, the `objectTrack` output contains `'State'`, `'StateCovariance'`, and `'TrackID'` information.

Data Types: `struct`

Track identifiers, specified as a real-valued vector. `trackIDs` elements match the tracks found in `tracks`.

Truth information, specified as an array of structures or objects. When using a `trackingScenario`, truth information can be obtained from the `platformPoses` object function.

Data Types: `struct`

Truth identifiers, specified as a real-valued vector. `truthIDs` elements match the truths found in `truths`.

Output Arguments

expand all

Position root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Velocity root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Acceleration root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Yaw rate root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Position average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Velocity average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Acceleration average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Yaw rate average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the `ErrorFunctionFormat` property to `'built-in'`.

Custom error metric outputs, returned as scalars. These errors are the output of the error estimation function specified in the `EstimationErrorFcn` property.

Dependencies

To enable these arguments, set the `ErrorFunctionFormat` property to `'custom'`.

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named `obj`, use this syntax:

`release(obj)`

expand all

 `cumulativeTrackMetrics` Cumulative metrics for recent tracks `cumulativeTruthMetrics` Cumulative metrics for recent truths `currentTrackMetrics` Metrics for recent tracks `currentTruthMetrics` Metrics for recent truths
 `release` Release resources and allow changes to System object property values and input characteristics `reset` Reset internal states of System object `isLocked` Determine if System object is in use `clone` Create duplicate System object

Examples

collapse all

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

`load trackmetricex tracklog truthlog`

Create objects to analyze assignment and error metrics.

```tam = trackAssignmentMetrics; tem = trackErrorMetrics;```

Create the output variables.

```posRMSE = zeros(numel(tracklog),1); velRMSE = zeros(numel(tracklog),1); posANEES = zeros(numel(tracklog),1); velANEES = zeros(numel(tracklog),1);```

Loop over all tracks to:

• Extract the tracks and ground truth at the i th tracker update.

• Analyze and retrieve the current track-to-truth assignment.

• Analyze instantaneous error metrics over all tracks and truths.

```for i=1:numel(tracklog) tracks = tracklog{i}; truths = truthlog{i}; [trackAM,truthAM] = tam(tracks, truths); [trackIDs,truthIDs] = currentAssignment(tam); [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ... tem(tracks,trackIDs,truths,truthIDs); end```

Show the track metrics table.

`trackMetricsTable(tam)`
```ans=4×15 table TrackID AssignedTruthID Surviving TotalLength DeletionStatus DeletionLength DivergenceStatus DivergenceCount DivergenceLength RedundancyStatus RedundancyCount RedundancyLength FalseTrackStatus FalseTrackLength SwapCount _______ _______________ _________ ___________ ______________ ______________ ________________ _______________ ________________ ________________ _______________ ________________ ________________ ________________ _________ 1 NaN false 1120 false 0 false 3 3 false 0 0 false 0 0 2 NaN false 1736 false 0 false 8 88 false 0 0 false 28 3 6 3 true 1138 false 0 false 4 314 false 1 28 false 0 2 8 2 true 662 false 0 false 2 29 false 1 169 false 28 0 ```

Show the truth metrics table.

`truthMetricsTable(tam)`
```ans=2×10 table TruthID AssociatedTrackID DeletionStatus TotalLength BreakStatus BreakCount BreakLength InCoverageArea EstablishmentStatus EstablishmentLength _______ _________________ ______________ ___________ ___________ __________ ___________ ______________ ___________________ ___________________ 2 8 false 2678 false 4 168 true true 56 3 6 false 2678 false 3 645 true true 84 ```

Plot the RMSE and ANEES error metrics.

```subplot(2,2,1) plot(posRMSE) title('Position Error') xlabel('tracker update') ylabel('RMSE (m)') subplot(2,2,2) plot(velRMSE) title('Velocity Error') xlabel('tracker update') ylabel('RMSE (m/s)') subplot(2,2,3) plot(posANEES) title('Position Error') xlabel('tracker update') ylabel('ANEES') subplot(2,2,4) plot(velANEES) title('Velocity Error') xlabel('tracker update') ylabel('ANEES')```

Show the current error metrics for each individual recorded track.

`currentTrackMetrics(tem)`
```ans=2×5 table TrackID posRMS velRMS posANEES velANEES _______ ______ ______ ________ ________ 6 44.712 20.988 0.05974 0.31325 8 129.26 12.739 1.6745 0.2453 ```

Show the current error metrics for each individual recorded truth object.

`currentTruthMetrics(tem)`
```ans=2×5 table TruthID posRMS velRMS posANEES velANEES _______ ______ ______ ________ ________ 2 129.26 12.739 1.6745 0.2453 3 44.712 20.988 0.05974 0.31325 ```

Show the cumulative error metrics for each individual recorded track.

`cumulativeTrackMetrics(tem)`
```ans=4×5 table TrackID posRMS velRMS posANEES velANEES _______ ______ ______ ________ ________ 1 117.69 43.951 0.58338 0.44127 2 129.7 42.8 0.81094 0.42509 6 371.35 87.083 4.5208 1.6952 8 130.45 53.914 1.0448 0.44813 ```

Show the cumulative error metrics for each individual recorded truth object.

`cumulativeTruthMetrics(tem)`
```ans=2×5 table TruthID posRMS velRMS posANEES velANEES _______ ______ ______ ________ ________ 2 258.21 65.078 2.2514 0.93359 3 134.41 48.253 0.96314 0.49183 ```
Introduced in R2018b