testStream

Create connection to event stream hosted by MATLAB with schema processing applied

This object requires Streaming Data Framework for MATLAB^® Production Server™.

Description

The testStream function creates a TestStream object, which you can use to test reading from and writing to event streams hosted by MATLAB. TestStream objects store events in MATLAB variables, which are exported to event streams based on the data type values specified by the ExportOptions property. TestStream applies the standard configurable schema processing when reading and writing timetable data. Use this object to test your schema management before deployment, without requiring a streaming service host. The data in a TestStream object disappears when you exit MATLAB.

Use TestStream to develop your streaming analytic function without having to connect to or consume resources from a streaming service such as Kafka^®. TestStream stores data using MATLAB memory space, so use this object with finite-size data sets. Because the data is stored in MATLAB, TestStream objects typically stream data faster than streams that have data stored on a network or file system.

Creation

Syntax

ts = testStream

ts = testStream(Name=Value)

Description

ts = testStream creates an object connected to an event stream hosted by MATLAB.

ts = testStream(Name=Value) specifies event stream options using one or more Name=Value arguments. Name can also be a property name, with Value as the corresponding value. You can specify several name-value arguments in any order as Name1=Value1,...,NameN=ValueN.

Input Arguments

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Duration — Timestamp span in event window

0 (default) | duration scalar

Timestamp span in the event window, specified as a duration scalar. Duration determines the events that the readtimetable function returns based on their timestamp. Duration specifies the difference between the last and first timestamps of events in the event window.

You can specify either the Duration property or the Rows property, but not both.

Example: Duration=minutes(1) specifies that each call to readtimetable returns a timetable that has one minute's worth of events, where the timestamp of the last event is no more than one minute later than the timestamp of the first event.

Data Types: duration

Rows — Number of events in event window

100 (default) | positive integer

Number of events in the event window, specified as a positive integer. Rows specifies the number of rows that a call to the readtimetable function returns. If there are less than the number of specified rows available for reading, then readtimetable times out and returns an empty timetable.

You can specify either the Duration property or the Rows property, but not both.

Example: Rows=500 specifies that each call to readtimetable returns a timetable with 500 rows.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

Properties

KeyVariable — Name of key variable

key (default) | string scalar | character vector

Name of the key variable in the event stream, specified as a string scalar or character vector.

Data Types: string | char

Name — Event stream name

string scalar | character vector

Name of the event stream, specified as a string scalar or character vector. This property is provided for compatibility with other stream objects. If you do not specify a name, the testStream function generates one when you create the TestStream object.

Example: CoolingFan

Data Types: string | char

WindowSize — Event window size

50 (default) | duration scalar | positive integer

This property is read-only.

Event window size, specified by a fixed amount of time (using the Duration argument) or a fixed number of messages (using the Rows argument).

Data Types: duration | single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

TimestampResolution — Unit of event timestamp

"Milliseconds" (default) | "Seconds" | "Minutes" | "Hours" | "Days"

Unit of event timestamp, specified as one of these values:

• "Milliseconds"

• "Seconds"

• "Minutes"

• "Hours"

• "Days"

Interpret the event timestamp as the number of corresponding units before or after the Unix epoch.

Data Types: string | char

ReadLimit — Wait strategy

"Size" (default) | "Time"

Strategy to wait for a response from the stream, specified as one of these values:

• "Size" — Client prioritizes filling the event window. Using this strategy, the client might wait longer than the RequestTimeout time period as long as it is still receiving the expected number of messages. The default number of messages is 50. If the client receives no messages within the RequestTimeout time period, it no longer waits.

• "Time" — Client strictly adheres to the RequestTimeout limit, even if it has not received the expected number of messages. RequestTimeout specifies the amount of time the stream object waits between receiving events. If the stream is actively receiving data, it does not time out during that operation.

Note

This object does not implement the ReadLimit property and does not have a RequestTimeout property. It is provided for compatibility with other stream connector objects. By setting the wait strategy in this object, you can more easily update your code to switch between objects that do implement this property, such as KafkaStream.

Import and Export Schema

ImportOptions — Rules for transforming stream events into MATLAB data

ImportOptions object

Rules for transforming stream events into MATLAB data, specified as an ImportOptions object. This object controls the import of stream events into MATLAB.

ExportOptions — Rules for transforming MATLAB data into stream events

ExportOptions object

Rules for transforming MATLAB data into stream events, specified as an ExportOptions object. This object controls the export of MATLAB data into streams.

PublishSchema — Flag to indicate whether export schema is written to output stream

true (default) | false

Flag to indicate whether the export schema is written to the output stream, specified as a logical scalar.

The schema is embedded in each event, which can significantly increase the size of the event. If downstream applications do not require the schema, set this flag to false to reduce the number of bytes in your stream.

Data Types: logical

Event Key and Body Encoding

BodyEncoding — Character encoding format for bits in event body

utf8 (default) | uint8 | utf16 | base64

Character encoding format used to interpret the bits in the event body, specified as one of the following:

• utf8 — UTF-8 encoding format

• utf16 — UTF-16 encoding format

• base64— Base 64 encoding format

• uint8 — Eight-bit unsigned binary bytes

This property determines the size and encoding of the bytes used in the event body, which are in the format specified by BodyFormat.

BodyFormat — Format of bytes in event body

JSON (default) | Array | Text | Binary

Format of bytes in event body, specified as one of the following:

• JSON — JSON string

• Array — MATLAB array

• Text — String data

• Binary — Binary data

Depending on the encoding specified by BodyEncoding, bytes can be larger than eight bits.

KeyByteOrder — Order for storing bits in event key

BigEndian (default) | LittleEndian | MatchHost | NotApplicable

Order for storing bits in the event key, specified as one of the following.

• LittleEndian — Least significant bit is stored first

• BigEndian — Most significant bit is stored first

• MatchHost— Bits are stored in the same order as is used by the host computer on which the streaming data framework is running

• NotApplicable — Not an integer key

This property is applicable only for integer keys and not applicable to floating point or text keys.

KeyEncoding — Character encoding format for bits in event key

uint8 (default) | utf16 | utf8 | base64

Character encoding format used to interpret the bits in an event key, specified as one of the following:

• utf8 — UTF-8 encoding format

• utf16 — UTF-16 encoding format

• base64— Base 64 encoding format

• uint8 — Eight-bit unsigned binary bytes

If KeyEncoding is utf8 or utf16, then the KeyType property must be text. If KeyEncoding is base64 or uint8, then KeyType must be one of the numeric encoding formats.

KeyType — Character encoding scheme for bytes in event key

text (default) | utf16 | int8 | uint8 | int16 | uint16 | int32 | uint32 | int64 | uint64 | single | double

Character encoding scheme used to interpret the bytes in an event key, specified as one of these values:

• uint8 — One-byte unsigned integer

• int8 — One-byte signed integer

• uint16 — Two-byte unsigned integer

• int16 — Two-byte signed integer

• uint32 — Four-byte unsigned integer

• int32 — Four-byte signed integer

• uint64 — Eight-byte unsigned integer

• int64 — Eight-byte signed integer

• single — Single-precision IEEE 754 floating point number

• double — Double-precision IEEE 754 floating point number

• text — String

If KeyType is text, then the KeyEncoding property must be either utf8 or utf16. If KeyType is any of the other numeric encoding formats, then KeyEncoding must be either base64 or uint8.

Object Functions

readtimetable	Read timetable from event stream
writetimetable	Write timetable to event stream
seek	Set read position in event stream
preview	Preview subset of events from event stream
identifyingName	Event stream name
detectImportOptions	Create import options based on event stream content
detectExportOptions	Create export options based on event stream content

Examples

Write and Preview Events for Test Stream

Create a TestStream object to read from and write events to an event stream hosted by MATLAB.

ts = testStream

t = 

  TestStream with properties:

                   Name: "8ee84682-dcb6-4dc7-a10a-873f80dc5f98"
          ImportOptions: "None"
          ExportOptions: "Source: function eventSchema"
          PublishSchema: "true"
             WindowSize: 100
            KeyVariable: "key"
            KeyEncoding: "uint8"
                KeyType: "text"
           KeyByteOrder: "BigEndian"
           BodyEncoding: "utf8"
             BodyFormat: "JSON"
              ReadLimit: "Size"
    TimestampResolution: "Milliseconds"

Write sample timetable data to the event stream.

load indoors
writetimetable(ts,indoors)

Preview data from the timetable. The timetable data has no key column, but writetimetable generates an empty key column in the stream by default. This makes it easier to transition your code from using a TestStream object to a KafkaStream object, which includes this key column to identify the event source.

preview(ts)

ans =

  8×3 timetable

         timestamp          Humidity    AirQuality    key
    ____________________    ________    __________    ___

    15-Nov-2015 00:00:24       36           80        "" 
    15-Nov-2015 01:13:35       36           80        "" 
    15-Nov-2015 02:26:47       37           79        "" 
    15-Nov-2015 03:39:59       37           82        "" 
    15-Nov-2015 04:53:11       36           80        "" 
    15-Nov-2015 06:06:23       36           80        "" 
    15-Nov-2015 07:19:35       36           80        "" 
    15-Nov-2015 08:32:47       37           80        "" 

Version History

Introduced in R2022b