- https://www.mathworks.com/help/matlab/ref/tic.html
- https://www.mathworks.com/help/matlab/ref/toc.html
set a maximum training time for training a PPO agent
32 views (last 30 days)
Show older comments
Danial Kazemikia
on 30 Jul 2024 at 0:26
In training process of a PPO RL agent, how can I make the code check the elapsed time and stop training if it exceeds the desired threshold. Suppose you want to stop training after a maximum of 30 minutes (1800 seconds).
0 Comments
Accepted Answer
Satwik
on 30 Jul 2024 at 8:05
Hi,
I understand that you want to limit the training process of your RL PPO agent based on the elapsed time. According to my knowledge and the documentation for ‘rlTrainingOptions’, there is no predefined option in the ‘StopTrainingCriteria’ property which can achieve this directly. However, a possible workaround is to use the custom stop criteria by specifying ‘StopTrainingValue’ as a function name or handle. Here is the documentation link for reference:
Below is an example code snippet demonstrating this approach:
% Set the maximum training time (in seconds)
maxTrainingTime = 1800; % 30 minutes
% Custom stop function
function stopTraining = customStopFunction(trainingStats)
persistent startTime;
if isempty(startTime)
startTime = tic;
end
elapsedTime = toc(startTime);
if elapsedTime > maxTrainingTime
stopTraining = true;
disp(['Stopping training after ', num2str(elapsedTime), ' seconds.']);
else
stopTraining = false;
end
end
% Set training options
trainingOptions = rlTrainingOptions(...
'MaxEpisodes', 10000, ...
'MaxStepsPerEpisode', 500, ...
'ScoreAveragingWindowLength', 100, ...
'Verbose', true, ...
'Plots', 'training-progress', ...
'StopOnError', 'off', ...
'SaveAgentCriteria', 'EpisodeReward', ...
'SaveAgentValue', 500, ...
'StopTrainingCriteria', 'Custom', ...
'StopTrainingValue', @customStopFunction);
% Train the agent
trainStats = train(agent, env, trainingOptions); % 'env' is the defined
% environment and 'agent' is the rlPPOAgent.
Here is the reference to the ‘tic’ and ‘toc’ functions used in the ‘customStopFunction’ to capture time:
I hope this gives you a direction for taking the next steps to achieve the desired result.
0 Comments
More Answers (0)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!