summarize
Summarize object detection performance metrics at data set and class level
Since R2024b
Syntax
Description
[
summarizes the object detection evaluation metrics across all classes of the data
set summaryDataset
,summaryClass
] = summarize(metrics
)summaryDataset
, and for each class in the data set
summaryClass
.
[
specifies the metrics category to return in the metrics summary.summaryDataset
,summaryClass
] = summarize(metrics
,MetricName=metricName
)
Examples
Summarize Object Detection Metrics
Load a table containing image file names and ground truth bounding box labels into the workspace. The first column contains the image file names, and the remaining columns contain the labeled bounding boxes.
data = load("vehicleTrainingData.mat");
trainingData = data.vehicleTrainingData;
Set the value of the dataDir
variable as the location where the vehicleTrainingData.mat
file is located. Load the test data into a local vehicle data folder.
dataDir = fullfile(toolboxdir("vision"),"visiondata"); trainingData.imageFilename = fullfile(dataDir,trainingData.imageFilename);
Create an imageDatastore
using the files from the table.
imds = imageDatastore(trainingData.imageFilename);
Create a boxLabelDatastore
using the label column from the table.
blds = boxLabelDatastore(trainingData(:,2:end));
Load a pretrained YOLO v2 object detector, trained to detect vehicles, into the workspace.
vehicleDetector = load("yolov2VehicleDetector.mat");
detector = vehicleDetector.detector;
Run the detector on the test images. Set the detection threshold to a low value to detect as many objects as possible. This enables you to evaluate the detector precision across the full range of recall values.
results = detect(detector,imds,Threshold=0.01);
Compute metrics for evaluating the performance of the object detector.
metrics = evaluateObjectDetection(results,blds);
Summarize the object detection metrics for the vehicle class using the summarize
object function.
[summaryDataset,summaryClass] = summarize(metrics); summaryClass
summaryClass=1×3 table
NumObjects APOverlapAvg AP0.5
__________ ____________ _______
vehicle 336 0.99096 0.99096
Input Arguments
metrics
— Object detection performance metrics
objectDetectionMetrics
object
Object detection performance metrics, specified as an objectDetectionMetrics
object.
metricName
— Metrics category to return
"all"
(default) | "AP"
| "LAMR"
| "AOS"
Metrics category to return in the metrics summary, specified as one of these options.
"all"
– Returns the average precision and mean average precision metric values, aggregated over each overlap threshold, along with all additional metrics."AP"
– Returns only the average precision across all classes at each overlap threshold as a numThresh-by-1 numeric vector, and the mean average precision (mAP) computed across all overlap thresholds. numThresh is the number of overlap thresholds specified by theOverlapThreshold
property of theobjectDetectionMetrics
object."LAMR"
– Returns only the log-average miss rate (LAMR) computed at each overlap threshold as a numThresh-by-1 numeric vector, and the mean of the log-average miss rate (mLAMR) computed across all overlap thresholds."AOS"
– Returns only the average orientation similarity (AOS) computed at each overlap threshold as a numThresh-by-1 array, and the mean of the average orientation similarity (mAOS) computed across all overlap thresholds.To specify
"AOS"
as the metric to return, your input data must contain rotated bounding boxes.
Output Arguments
summaryDataset
— Metrics summary over data set
table
Metric summary over the data set, returned as a 1-by-numMetrics table, where numMetrics is the number of specified metrics.
The columns of the table depend on the value of the
metricName
argument.
"all"
—NumObjects
mAPOverlapAvg
mAP
mLAMROverlapAvg
mLAMR
mAOSOverlapAvg
mAOS
Number of objects in the ground truth data, returned as a positive integer.
Mean average precision (mAP) averaged over all thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.Mean log-average miss rate (mLAMR) averaged over all classes and all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Mean log-average miss rate (mLAMR) averaged over all classes, computed at all overlap thresholds, specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector.Mean average orientation similarity (mAOS) averaged over all classes, and averaged over all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Mean average orientation similarity (mAOS) averaged over all classes, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector.To return the
"mAOSOverlapAvg"
and"mAOS"
columns, your input data must contain rotated bounding boxes."AP"
—NumObjects
mAPOverlapAvg
mAP
Number of objects in the ground truth data, returned as a positive integer.
Mean average precision (mAP) averaged over all thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Mean average precision (mAP), or average precision averaged over all classes, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds."LAMR"
—mLAMROverlapAvg
mLAMR
Mean log-average miss rate (mLAMR) averaged over all classes and all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a positive scalar.Mean log-average miss rate (mLAMR) averaged over all classes, computed at all overlap thresholds, specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector."AOS"
—mAOSOverlapAvg
mAOS
Mean average orientation similarity (mAOS) averaged over all classes and all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Mean average orientation similarity (mAOS) averaged over all classes, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector.
summaryClass
— Metrics summary per class
table
Metrics summary per class, returned as a numClasses-by-numMetrics table, where numClasses is the number of classes, and numMetrics is the number of specified metrics.
The columns of the table depend on the value of the
metricName
argument.
"all"
—NumObjects
APOverlapAvg
AP
LAMROverlapAvg
LAMR
AOSOverlapAvg
AOS
Number of objects in the ground truth data for each class, returned as a positive integer.
Average precision (AP) for each class averaged over all thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Average precision (AP) for each class, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds.Log-average miss rate (LAMR) for each class, averaged over all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
.Log-average miss rate (LAMR) for each class, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector.Average orientation similarity (AOS) for each class, averaged over all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Average orientation similarity (AOS) for each class, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector.To return the
"AOSOverlapAvg"
and"AOS"
columns, your input data must contain rotated bounding boxes."AP"
—NumObjects
APOverlapAvg
AP
Number of objects in the ground truth data for each class, returned as a positive integer.
Average precision (AP) for each class averaged over all thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Average precision (AP) for each class, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector. numThresh is the number of overlap thresholds."LAMR"
—LAMROverlapAvg
LAMR
Log-average miss rate (LAMR) for each class, averaged over all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
.Log-average miss rate (LAMR) for each class, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector."AOS"
—AOSOverlapAvg
AOS
Average orientation similarity (AOS) for each class, averaged over all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numeric scalar.Average orientation similarity (AOS) for each class, computed at all overlap thresholds specified by the
OverlapThreshold
property ofmetrics
, returned as a numThresh-by-1 numeric vector.
Version History
Introduced in R2024b
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)