Main Content

matlab.io.datastore.SimulationDatastore

Datastore for inputs and outputs of Simulink models

expand all in page

Description

A matlab.io.datastore.SimulationDatastore object enables a Simulink® model to interact with big data. When individual input signals are too large to fit into memory, store data to persistent storage in a MAT-file and refer to the data through a SimulationDatastore object. The data from the SimulationDatastore object loads into the simulation incrementally in chunks that fit into memory. See Work with Big Data for Simulations.

A SimulationDatastore object refers to big simulation data for one signal stored in a MAT-file. If the MAT-file stores simulation data for a bus, a SimulationDatastore object refers to the data for one leaf element in the bus.

Note

You cannot create a SimulationDatastore representation for Dataset elements that contain array data.

To analyze the datastore data, you can use the functions and properties of the SimulationDatastore object as well as functions such as the tall function. For more information, see Getting Started with Datastore.

Creation

To create a SimulationDatastore object, first create a Simulink.SimulationData.DatasetRef object. The DatasetRef object references a Simulink.SimulationData.Dataset object, which is stored in a MAT-file, that contains your element of interest. Then, use curly braces or the getAsDatastore function to get a SimulationDatastore representation of that element from the referenced Dataset object.

When the Dataset element of interest is a timeseries or timetable object, the getAsDatastore function returns a SimulationDatastore object. Otherwise, the SimulationDatastore object exists in the Values property of the returned Simulink.SimulationData.Signal, Simulink.SimulationData.State, or similar object.

For example, suppose you configure a model to log signal data to persistent storage by selecting the Log Dataset data to file configuration parameter in the Configuration Parameters > Data Import/Export pane and simulate that model. All logged simulation data that uses the Dataset format is logged to a MAT-file with the default name out. In the MAT-file, signal logging data is saved in a Simulink.SimulationData.Dataset object with the default variable name logsout. Create a DatasetRef object that references the signal logging data in the MAT-file. Then, use curly braces to get a SimulationDatastore object for the first element in the referenced Dataset object. The SimulationDatastore object exists in the Values property of the returned Simulink.SimulationData.Signal object.

sigLogRef = Simulink.SimulationData.DatasetRef("out.mat","logsout");
firstSig = sigLogRef{1}
firstSig = 

  Simulink.SimulationData.Signal
  Package: Simulink.SimulationData

  Properties:
              Name: 'x1'
    PropagatedName: ''
         BlockPath: [1x1 Simulink.SimulationData.BlockPath]
          PortType: 'outport'
         PortIndex: 1
            Values: [1×1 matlab.io.datastore.SimulationDatastore]

Properties

expand all

This property is read-only.

Name and path of file that contains the big data, returned as a character vector.

This property is read-only.

Number of samples (time steps) in the datastore, returned as a positive integer. The readall function extracts this many samples from the big data.

Amount of data to read at a time, in number of samples (time steps), specified as a positive integer. The read function extracts this many samples from the big data.

Object Functions

hasdata Determine whether data is available to read
isPartitionableDetermine whether datastore is partitionable
isShuffleableDetermine whether datastore is shuffleable
resetReset datastore to initial position
readallRead all data in datastore
readRead chunk of data in datastore
preview Return subset of data from datastore
progressReturn percentage of data read from datastore

Examples

collapse all

Open Live Script

Open the SineWave model, which contains a Sine Wave block connected to an Outport block.

mdl = "SineWave";
open_system(mdl)

The SineWave model

Configure the model to log output data to persistent storage and simulate a model.

  1. In the Configuration Parameters > Data Import/Export pane, select Log Dataset data to file. Then, click OK.

  2. Click Run to simulate the model.

Alternatively, you can configure the model to log Dataset data to a file and simulate the model programmatically. By default, the data is saved to a MAT-file named out.mat.

sim(mdl,"LoggingToFile","on");

Create a DatasetRef object for the logged output data from the SineWave model. 

sigLogRef = Simulink.SimulationData.DatasetRef("out.mat","yout");

Use the getAsDatastore function to create a SimulationDatastore object for the sineSig signal. The SimulationDatastore object exists in the Values property of the returned Simulink.SimulationData.Signal object.

sineSig_dst = getAsDatastore(sigLogRef,"sineSig")
sineSig_dst = 
  Simulink.SimulationData.Signal
  Package: Simulink.SimulationData

  Properties:
              Name: 'sineSig'
    PropagatedName: ''
         BlockPath: [1x1 Simulink.SimulationData.BlockPath]
          PortType: 'inport'
         PortIndex: 1
            Values: [1x1 matlab.io.datastore.SimulationDatastore]

The model MultByTwo contains a Gain block that multiplies input data by 2 and logs the data using an Outport block.

The MultByTwo model

Load the data referenced from the SineWave model into the MultByTwo model. Because the Values property of the sineSig_dst Signal object is a SimulationDatastore object, the signal data streams into your model. 

mdl2 = "MultByTwo";
open_system(mdl2)
multByTwoOut = sim(mdl2,"LoadExternalInput","on","ExternalInput","sineSig_dst");

The Dashboard Scope block in the MultByTwo model shows that the input streamed from the SineWave model is a sine wave with an amplitude of 1, and the output is a sine wave with an amplitude of 2.

The Dashboard Scope block plot

Open Live Script

You can log big data from a simulation and inspect and analyze portions of that data by interacting with a matlab.io.datastore.SimulationDatastore object.

Log Big Data from Model

Open the model sldemo_fuelsys.

mdl = "sldemo_fuelsys";
open_system(mdl)

Select Configuration Parameters > Data Import/Export > Log Dataset data to file to log data to persistent storage. Alternatively, you can log data using Dataset format to a MAT-file instead of the workspace programmatically. 

set_param(mdl,"LoggingToFile","on")

Simulate the model.

sim(mdl);

The MAT-file out.mat appears in your current folder. Logged signal data is stored in the MAT-file with the variable name sldemo_fuelsys_out.

Create a DatasetRef object that refers to the logged signal data.

DSRef = Simulink.SimulationData.DatasetRef("out.mat","sldemo_fuelsys_output");

Preview Big Data

Use curly braces to return a SimulationDatastore representation of the fuel signal, which is the tenth element in the DatasetRef object DSRef. The SimulationDatastore object exists in the Values property of the returned Signal object.

SimDataSig = DSRef{10};
DStore = SimDataSig.Values;

Use the preview function to inspect the first ten samples of logged data for the fuel signal.

preview(DStore)
ans=10×1 timetable
         Time          Data 
    ______________    ______

    0 sec             1.2855
    0.00056195 sec    1.2855
    0.0033717 sec     1.2855
    0.01 sec          1.2398
    0.02 sec           1.199
    0.03 sec          1.1628
    0.04 sec          1.1309
    0.043098 sec      1.1309
    0.043098 sec      1.1309
    0.043098 sec      1.1309

Inspect Specific Sample

Suppose you want to inspect the 603rd sample of logged fuel data. Set the ReadSize property of DStore to a number that, considering memory resources, your computer can tolerate. For example, set ReadSize to 200.

DStore.ReadSize = 200;

Read from the datastore three times. Each read operation advances the reading position by 200 samples.

read(DStore);
read(DStore);
read(DStore);

Now that you are close to the 603rd sample, you can set ReadSize to a smaller number to make the target sample easier to find. For example, set ReadSize to 5.

DStore.ReadSize = 5;

Read from the datastore again. The third sample of read data is the 603rd sample in the datastore.

read(DStore)
ans=5×1 timetable
      Time       Data 
    ________    ______

    5.83 sec    1.6609
    5.84 sec    1.6733
    5.85 sec    1.6831
    5.86 sec     1.691
    5.87 sec    1.6975

Inspect Earlier Sample

You can also inspect an earlier sample. For example, inspect the 403rd sample of logged fuel data. Due to previous read operations, the datastore now reads starting from the 606th sample.

Use the reset function to reset DStore. Then, read from the first sample up to the 403rd sample.

reset(DStore);

Set ReadSize to 200.

DStore.ReadSize = 200;

Read from the datastore twice to advance the read position to the 401st sample.

read(DStore);
read(DStore);

Set ReadSize to 5 and read from the datastore. Now, the third sample of read data is the 403rd sample in the datastore.

DStore.ReadSize = 5;
read(DStore)
ans=5×1 timetable
      Time       Data  
    ________    _______

    3.86 sec     0.9895
    3.87 sec    0.98253
    3.88 sec    0.97559
    3.89 sec    0.96867
    3.9 sec     0.96178

Extract Multiple Samples

You can also use the read function to extract multiple samples. For example, extract samples 1001 through 1020.

Reset the datastore. Then, advance to sample 1001 by setting the ReadSize property to 200 and reading the datastore five times.

reset(DStore)

DStore.ReadSize = 200;
for i = 1:5
    read(DStore);
end

Set the ReadSize to 20 to extract 20 samples from the datastore.

DStore.ReadSize = 20;

Extract samples 1001 through 1020. Store the extracted data in a variable named targetSamples.

targetSamples = read(DStore)
targetSamples=20×1 timetable
      Time       Data 
    ________    ______

    9.77 sec      1.68
    9.78 sec    1.6856
    9.79 sec    1.6905
    9.8 sec     1.6948
    9.81 sec    1.6812
    9.82 sec    1.6714
    9.83 sec    1.6642
    9.84 sec     1.659
    9.85 sec    1.6553
    9.86 sec    1.6527
    9.87 sec    1.6684
    9.88 sec    1.6806
    9.89 sec    1.6904
    9.9 sec     1.6982
    9.91 sec    1.7047
    9.92 sec    1.7101
      ⋮

Find Maximum Value of Data in Datastore

Use the hasdata function as the condition for a while loop to incrementally analyze the data in chunks of 200 samples.

reset(DStore);
DStore.ReadSize = 200;
runningMax = [];
while hasdata(DStore)
    tt = read(DStore);
    rawChunk = tt.Data;
    runningMax = max([rawChunk; runningMax]);
end

The variable runningMax stores the maximum value in the entire datastore.

runningMax
runningMax = 
1.7291

Limitations

  • SimulationDatastore does not support using a parallel pool with Parallel Computing Toolbox™ installed. To analyze data using tall arrays or run MapReduce algorithms, set the global execution environment to be the local MATLAB® session using mapreducer. 

    mapreducer(0)
    For information about controlling parallel resources, see Run mapreduce on a Parallel Pool (Parallel Computing Toolbox).

  • You cannot use a MATLAB tall variable as simulation input data.

Version History

Introduced in R2017a

See Also

Topics