How can I deploy very large datasets onto embedded hardware while avoiding out-of-memory errors?

I’m attempting to deploy a massive time series dataset (along with my Simulink model) onto my TI development board. To do this, I'm using Embedded Coder and the C2000 Microcontroller Blockset in Simulink R2024b. Unfortunately, there’s not enough available memory on the board to fit the entire dataset, and so I keep getting out-of-memory errors when clicking "Monitor & Tune" or "Build, Deploy, & Start" from the Hardware Tab:

program will not fit into available memory, or the section contains a call
site that requires a trampoline that can't be generated for this section.
run placement with alignment/blocking fails for section ".ebss" size 0x3801c
page 1. Available memory ranges:
RAMGS_DATA size: 0x7000 unused: 0x65ab max hole: 0x65ab
RAMLS_DATA size: 0x1000 unused: 0x1000 max hole: 0x1000

How can I make the most out of the memory available on my board to fit the dataset and avoid these out-of-memory errors?

Answer 1

[MathWorks Support Team]

There are many potential options for using the memory available on the board more efficiently. Before listing them, here is something important to consider: For all embedded hardware (microcontrollers, TI boards, etc.), the memory is almost never one giant, continuous chunk. Instead, memory (whether it be flash memory, RAM, etc.) is split up into various discrete sections, or “holes”, which are usually somewhere around 8-64KB long. Simply put, this means that the dataset must either be split up as well, or it must be smaller than the largest available “max hole”.

With this under consideration, here are some potential options to consider:

1. Cast the dataset to a smaller datatype if possible (e.g. “double” -> “single”, “uint16” -> “uint8”, etc.). If the situation allows for less numerical precision/bandwidth than the current datatype is providing, this option can reduce the memory usage by half or more. Like all of these suggestions, this might require making some minor adjustments to the Simulink model. For more details on casting datatypes, see the following link: "cast" Function
2. Interpolate the dataset by using blocks such as “Prelookup” and “Interpolation Using Prelookup”. Using interpolation won’t increase the amount of available memory on the hardware, but it could allow you to reach similar levels of accuracy with far fewer data points. Depending on the situation and how these blocks are used, this option could reduce the memory requirement by several orders of magnitude. Details on these blocks (and how to use them) are listed here: "Prelookup" Block, "Interpolation Using Prelookup" Block
3. If using a time-based dataset, remove the time vector entirely and use a fixed time rate instead. Depending on the situation, this could significantly reduce the memory requirement. First, use the “resample” (or similar) function to change the time data from non-uniform to uniform, if applicable. Then, feed the resulting data into the model, without including time. However, keep in mind that the model must also be updated to the same chosen time rate. Please see more details on how to resample data here: "resample" Function
4. Split the dataset up into several different chunks, fitting each chunk into a separate discrete section of memory. This would involve incorporating some form of logic into the model in order to index through the chunks. Also, it could require mapping each chunk of data to a specific section of memory manually using Simulink Coder / Code Generation. However, it could increase the amount of available memory substantially.
5. If none of these strategies are sufficient, consider upgrading the hardware to obtain more memory.

Note: If the embedded hardware has more flash memory than RAM, it could be a good idea to mark the dataset as a “Const” Simulink parameter. This will ensure that it falls into flash memory (instead of RAM). See the following link for an example on how to do this: Type Qualifiers Example

You can also check the “.MAP” file to ensure that the dataset is placed into a “FLASH” region.