SceneLabelData

Ground truth data for scene labels

Description

The SceneLabelData object stores ground truth data for scene label definitions defined in a groundTruthMultisignal object.

Creation

When you export a groundTruthMultisignal object from a Ground Truth Labeler app session, the SceneLabelData property of the exported object stores the scene labels as a SceneLabelData object. To create a SceneLabelData object programmatically, use the vision.labeler.labeldata.SceneLabelData function (described here).

Syntax

sceneLabelData = vision.labeler.labeldata.SceneLabelData(labelNames,labelData)

Description

sceneLabelData = vision.labeler.labeldata.SceneLabelData(labelNames,labelData) creates an object containing scene label data for multiple signals. The created object, sceneLabelData, contains properties with the scene label names listed in labelNames. These properties store the corresponding scene label data specified by labelData.

Input Arguments

labelNames — Scene label names

string array

Scene label names, specified as a string array. Specify the names of all scene labels present in the groundTruthMultisignal object you are creating. You can get the scene label names from an existing groundTruthMultisignal object by accessing the LabelDefinitions property of that object. Use this code and replace gTruth with the name of a groundTruthMultisignal object variable.

isSceneLabel = gTruth.LabelDefinitions.LabelType == 'Scene';
gTruth.LabelDefinitions.Name(isSceneLabel)

In an exported groundTruthMultisignal object, the SceneLabelData object contains a label data property for every scene label, even if some scene labels do not have label data.

The properties of the created SceneLabelData object have the names specified by labelNames.

Example: ["sunny" "rainy" "urban" "rural"]

labelData — Scene label data for each label

cell array of duration matrices

Scene label data for each label, specified as a cell array of duration matrices. Each matrix in the cell array contains data for the scene label in the corresponding position of the labelNames input. The SceneLabelData object stores each matrix in a property that has the same name as that signal.

Each scene label matrix is of size N-by-2. Each row in this matrix corresponds to a time range for which that scene label has been applied. N is the number of time ranges. Rows in the matrix are of the form [rangeStart, rangeEnd], where rangeStart and rangeEnd specify the start and end of a time range for an applied scene label.

Row elements are of type duration and must be within the range of the minimum and maximum of all the timestamps in the groundTruthMultisignal object. If a scene label is not applied, then specify an empty matrix.

Example: seconds([0 5; 10 20]) specifies a duration matrix corresponding to one scene label in a groundTruthMultisignal object. Units are in seconds. The scene label has been applied from 0 to 5 seconds and again from 10 to 20 seconds, across all signals in the object. Specify this matrix as part of a cell array containing matrices for additional scene labels.

Properties

SceneLabelName1,...,SceneLabelNameN — Scene label data for each label (as separate properties)

duration matrices

Scene label data, specified as duration matrices. The SceneLabelData object contains one property per scene label definition, where each property contains a duration matrix of scene label data corresponding to that scene label.

When exporting a SceneLabelData object from a Ground Truth Labeler app session, the property names correspond to the scene label names stored in the LabelDefinitions property of the exported groundTruthMultisignal object.

When creating a SceneLabelData object programmatically, the labelNames and labelData input arguments define the property names and values of the created object.

Suppose you want to create a groundTruthMultisignal object containing scene labels that describe whether the scene is sunny, rainy, urban, or rural. Specify the scene labels in a string array, labelNames.

labelNames = ["sunny" "rainy" "urban" "rural"];

Store the label data for each scene label in a cell array of matrices, labelData. Each matrix contains the data for the corresponding scene label in labelNames.

labelData = {sunnyData,rainyData,urbanData,ruralData}

  1×4 cell array

    {1×2 duration}    {2×2 duration}    {0×0 duration}    {4×2 duration}

The SceneLabelData object, sceneData, stores this data in the property with the corresponding signal name. You can specify sceneData in the SceneLabelData property of a groundTruthMultisignal object.

sceneData = vision.labeler.labeldata.SceneLabelData(labelNames,labelData)

sceneData = 

  SceneLabelData with properties:

    rainy: [2×2 duration]
    sunny: [0 sec    10.15 sec]
    rural: [4×2 duration]
    urban: [0×0 duration]

Object Functions

labelDefinitionsAtTime	Get scene label definition names at specified timestamp
labelDataAtTime	Get scene label data at specified timestamps

Examples

Create Ground Truth from Multiple Signals

Create ground truth data for a video signal and a lidar point cloud sequence signal that captures the same driving scene. Specify the signal sources, label definitions, and ROI and scene label data.

Create the video data source from an MP4 file.

sourceName = '01_city_c2s_fcw_10s.mp4';
sourceParams = [];
vidSource = vision.labeler.loading.VideoSource;
vidSource.loadSource(sourceName,sourceParams);

Create the point cloud sequence source from a folder of point cloud data (PCD) files.

pcSeqFolder = fullfile(toolboxdir('driving'),'core','drivingdata','lidarSequence');
addpath(pcSeqFolder)
load timestamps.mat
rmpath(pcSeqFolder)

lidarSourceData = load(fullfile(pcSeqFolder,'timestamps.mat'));

sourceName = pcSeqFolder;
sourceParams = struct;
sourceParams.Timestamps = timestamps;

pcseqSource = vision.labeler.loading.PointCloudSequenceSource;
pcseqSource.loadSource(sourceName,sourceParams);

Combine the signal sources into an array.

dataSource = [vidSource pcseqSource]

dataSource = 

  1×2 heterogeneous MultiSignalSource (VideoSource, PointCloudSequenceSource) array with properties:

    SourceName
    SourceParams
    SignalName
    SignalType
    Timestamp
    NumSignals

Create a table of label definitions for the ground truth data by using a labelDefinitionCreatorMultisignal object.

• The Car label definition appears twice. Even though Car is defined as a rectangle, you can draw rectangles only for image signals, such as videos. The labelDefinitionCreatorMultisignal object creates an additional row for lidar point cloud signals. In these signal types, you can draw Car labels as cuboids only.

• The label definitions have no descriptions and no assigned colors, so the Description and LabelColor columns are empty.

• The label definitions have no assigned groups, so for all label definitions, the corresponding cell in the Group column is set to 'None'.

• Road is a pixel label definition, so the table includes a PixelLabelID column.

• No label definitions have sublabels or attributes, so the table does not include a Hierarchy column for storing such information.

ldc = labelDefinitionCreatorMultisignal;
addLabel(ldc,'Car','Rectangle');
addLabel(ldc,'Truck','ProjectedCuboid');
addLabel(ldc,'Lane','Line');
addLabel(ldc,'Road','PixelLabel');
addLabel(ldc,'Sunny','Scene');
labelDefs = create(ldc)

labelDefs =

  7×7 table

      Name       SignalType       LabelType        Group      Description    LabelColor    PixelLabelID
    _________    __________    _______________    ________    ___________    __________    ____________

    {'Car'  }    Image         Rectangle          {'None'}       {' '}       {0×0 char}    {0×0 double}
    {'Car'  }    PointCloud    Cuboid             {'None'}       {' '}       {0×0 char}    {0×0 double}
    {'Truck'}    Image         ProjectedCuboid    {'None'}       {' '}       {0×0 char}    {0×0 double}
    {'Lane' }    Image         Line               {'None'}       {' '}       {0×0 char}    {0×0 double}
    {'Lane' }    PointCloud    Line               {'None'}       {' '}       {0×0 char}    {0×0 double}
    {'Road' }    Image         PixelLabel         {'None'}       {' '}       {0×0 char}    {[       1]}
    {'Sunny'}    Time          Scene              {'None'}       {' '}       {0×0 char}    {0×0 double}

Create ROI label data for the first frame of the video.

numVideoFrames = numel(vidSource.Timestamp{1});
carData = cell(numVideoFrames,1);
laneData = cell(numVideoFrames,1);
truckData = cell(numVideoFrames,1);
carData{1} = [304 212 37 33];
laneData{1} = [70 458; 311 261];
truckData{1} = [309,215,33,24,330,211,33,24];
videoData = timetable(vidSource.Timestamp{1},carData,laneData, ...
                      'VariableNames',{'Car','Lane'});

Create ROI label data for the first point cloud in the sequence.

numPCFrames = numel(pcseqSource.Timestamp{1});
carData = cell(numPCFrames, 1);
carData{1} = [27.35 18.32 -0.11 4.25 4.75 3.45 0 0 0];
lidarData = timetable(pcseqSource.Timestamp{1},carData,'VariableNames',{'Car'});

Combine the ROI label data for both sources.

signalNames = [dataSource.SignalName];
roiData = vision.labeler.labeldata.ROILabelData(signalNames,{videoData,lidarData})

roiData = 

  ROILabelData with properties:

                lidarSequence: [34×1 timetable]
    video_01_city_c2s_fcw_10s: [204×2 timetable]

Create scene label data for the first 10 seconds of the driving scene.

sunnyData = seconds([0 10]);
labelNames = ["Sunny"];
sceneData = vision.labeler.labeldata.SceneLabelData(labelNames,{sunnyData})

sceneData = 

  SceneLabelData with properties:

    Sunny: [0 sec    10 sec]

Create a ground truth object from the signal sources, label definitions, and ROI and scene label data. You can import this object into the Ground Truth Labeler app for manual labeling or to run a labeling automation algorithm on it. You can also extract training data from this object for deep learning models by using the gatherLabelData function.

gTruth = groundTruthMultisignal(dataSource,labelDefs,roiData,sceneData)

gTruth = 

  groundTruthMultisignal with properties:

          DataSource: [1×2 vision.labeler.loading.MultiSignalSource]
    LabelDefinitions: [7×7 table]
        ROILabelData: [1×1 vision.labeler.labeldata.ROILabelData]
      SceneLabelData: [1×1 vision.labeler.labeldata.SceneLabelData]

Tips

• To create a groundTruthMultisignal object containing ROI label data but no scene label data, specify the SceneLabelData property as an empty array. To create this array, at the MATLAB^® command prompt, enter this code.

  sceneData = vision.labeler.labeldata.SceneLabelData.empty

Version History

Introduced in R2020a