- Limit the size of the experience replay buffer: The experience replay buffer can consume a significant amount of memory during training. You can reduce its size by adjusting the BufferSize parameter when creating the rlDQNAgent or rlDDPGAgent object. A smaller buffer size will require less memory but may affect learning performance.
- Reduce the training batch size: The batch size determines how many experiences are processed in each training iteration. A smaller batch size will reduce the memory footprint but may slow down training. You can adjust the MiniBatchSize parameter when creating the agent to decrease the batch size.
- Lower the resolution of observations: If the observations in your environment have high resolution, consider reducing their resolution. This can be achieved by downsampling or compressing the observations before feeding them into the agent. Lower resolution observations require less memory.
- Restart MATLAB periodically: As you mentioned, restarting MATLAB releases the memory consumed during training. If it's feasible for your workflow, you can periodically restart MATLAB after a certain number of training iterations or episodes to reclaim the memory. You can use the exit function to gracefully terminate MATLAB.
When training reinforcement learning agents, the C drive is fully occupied
22 views (last 30 days)
Show older comments
During the process of using the reinforcement learning toolbox to train agents, the occupancy of the C drive increased and eventually became full. I found that the virtual memory in the training process was very large (nearly 50G) by browsing the hidden files in the C disk. After restarting MATLAB, this part of memory will be released again.The training environment is built through Simscape.
I want to know what methods can be used to avoid the increasing usage of the C drive as the training duration increases. Because of this issue, I have to always terminate the training process.
0 Comments
Accepted Answer
Shaik
on 14 May 2023
Hi Yunyi,
When training reinforcement learning agents using the reinforcement learning toolbox in MATLAB, it's important to manage the memory usage to avoid filling up the C drive. Here are a few methods you can try to mitigate the increasing disk space usage during training:
Change the temporary directory: MATLAB uses a temporary directory for storing intermediate data during training. By default, this directory is set to the system's temporary folder, which is often located on the C drive. You can change the temporary directory to a different location, such as a different drive with more available space. Use the tempdir function in MATLAB to set a different temporary directory.
newTempDir = 'D:\Temp'; % Replace with the desired temporary directory
matlab.internal.getAndSetTemporaryDirectory(newTempDir);
These methods should help mitigate the increasing disk space usage during training. It's important to find the right balance between memory usage and training performance for your specific application.
2 Comments
Tobias Schindler
on 3 Jul 2023
It seems a bit worrisome that "restart Matlab" is a common solution to these kind's of memory leaks. Maybe some staff could provide some additional input here.
More Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!