Load Ground Truth Signals to Label

Step 1 of 3 in Get Started with Ground Truth Labelling

The Ground Truth Labeler app provides options for labeling two types of signals.

Image signals are image-based. You can load these signals from sources such as videos or image sequences.
Point cloud signals are lidar-based. You can load these signals from sources such as a sequence of point cloud files.

In this example, you load a video and a point cloud sequence into the app. These signals are taken from a camera sensor and a lidar sensor mounted to a vehicle. The signals represent the same driving scene.

Load Timestamps

Load the timestamps for the point cloud sequence. The timestamps are a duration vector that is in the same folder as the sequence. To load the timestamps, you must temporarily add this folder to the MATLAB^® search path.

pcSeqFolder = fullfile(toolboxdir('driving'),'drivingdata','lidarSequence');
addpath(pcSeqFolder)
load timestamps.mat
rmpath(pcSeqFolder)

The app also provides an option to specify timestamps for video sources. The video used in this example does not have a separate timestamps file, so when you load the video, you can read the timestamps directly from the video source.

Open Ground Truth Labeler App

To open the Ground Truth Labeler app, at the MATLAB command prompt, enter this command.

groundTruthLabeler

The app opens to an empty session.

Alternatively, you can open the app from the Apps tab, under Automotive.

Load Signals from Data Sources

The Ground Truth Labeler app enables you to load signals from multiple types of data sources. In the app, a data source is a file or folder containing one or more signals to label.

For the video, the data source is an MP4 file that contains a single video.
For the point cloud sequence, the data source is a folder containing a sequence of point cloud data (PCD) files. Together, these files represent a single point cloud sequence.

Other data sources, such as rosbags, can contain multiple signals that you can load. For more details on the relationship between sources and signals, see Sources vs. Signals in Ground Truth Labeling.

Load Video

Load the video into the app.

On the app toolstrip, click Import > Add Signals.
The Add/Remove Signal dialog box opens with the Source Type parameter set to Video and the Timestamps parameter set to From File.
In the File Name parameter, browse for this video file. <matlabroot> is the full path to your MATLAB installation folder, as returned by the matlabroot function.
```
<matlabroot>\toolbox\driving\drivingdata\01_city_c2s_fcw_10s.mp4
```
Click Add Source. The video loads into the app, and the app reads the timestamps directly from the video. The source table displays the information about the video data source.

Load Point Cloud Sequence

Load the point cloud sequence into the app.

With the Add/Remove Signal dialog box still open and the video loaded, set the Source Type parameter to Point Cloud Sequence. The dialog box displays new options specific to loading point cloud sequences.
In the Folder Name parameter, browse for the lidarSequence folder, which contains the sequence of point cloud data (PCD) files to load.
```
<matlabroot>\toolbox\driving\drivingdata\lidarSequence
```
Set the Timestamps parameter to From Workspace. In the Import From Workspace dialog box, select the timestamps variable that you loaded for the point cloud sequence. Click OK.
Click Add Source. The point cloud sequence loads into the app, and the app reads the timestamps from the timestamps variable. The source table displays the information about the data source for the point cloud sequence.

Verify Information About Loaded Signals

The table at the bottom of the Add/Remove Signal dialog box displays information about the loaded signals. Verify that the table displays this information for the loaded signals.

The Signal Name column displays the signal names generated by the app. For the video, the signal name is the file name of the data source with the prefix video_ and with no file extension. For the point cloud sequence, the signal name is the name of the source folder.
The Source column displays the full file paths to the signal data sources.
The Signal Type column displays the type of each signal. The video is of type Image. The point cloud sequence is of type Point Cloud.
The Time Range column displays the duration of the signals based on the loaded timestamp data. Both signals are approximately 10 seconds long.

For the point cloud sequence, if you left Timestamps set to Use Default, then the Time Range value for the sequence ranges from 0 to 33 seconds. This range is based on the 34 PCD files in the folder. By default, the app sets the timestamps of a point cloud sequence to a duration vector from 0 to the number of valid point cloud files minus 1. Units are in seconds. If this issue occurs, in the table, select the check box for the point cloud sequence row. Then, click Delete Selected, load the signal again, and verify the signal information again.

After verifying that the signals loaded correctly, click OK. The app loads the signals and opens to the first frame of the last signal added, which for this example is the point cloud sequence.

Labeling window with tabs for the video and point cloud sequence. The first frame of the point cloud sequence is displayed.

Configure Signal Display

When you first load the signals, the app displays only one signal at a time. To display the signals side-by-side, first, on the Label tab of the app toolstrip, click Display Grid. Then, move the pointer to select a 1-by-2 grid and click the grid.

Display Grid configured to display signals in a 1-by-2 grid

The video and point cloud sequence display side-by-side.

Labeling window displayed in a 1-by-2 grid. The video is on the left and the point cloud sequence is on the right.

To view the video and point cloud sequence together, in the slider below the signals, click the Play button . The video plays more smoothly than the point cloud sequence because the video has more frames over approximately the same amount of time and therefore a higher frame rate.

By default, the app plays all frames from the signal with the highest frame rate. This signal is called the main signal. For all other signals, the app displays the frame that is time-aligned with the currently displaying frame of the main signal. To configure which signal is the main signal, use the options in the Playback Control Settings dialog box. To open this dialog box, below the slider, click the clock settings button . For more details about using these options to control the display of signal frames, see Control Playback of Signal Frames for Labeling.

After loading the signal and viewing the frames, you can now create label definitions and label the data, as described in Label Ground Truth for Multiple Signals.