Hi,
To apply actions to multiple elements (7, 10, 13, and 16) in the "mult" array, you can use a multi-dimensional action space. Each dimension of this space represents the action to be applied at one of the specified elements. Given that each action can be one of 32 possible states for a single element, when dealing with 4 elements independently, the total number of combinations becomes (32^4 = 1,048,576). Then you'll need to encode and decode the actions in your environment's step function.
Encoding Actions:
You can encode the actions as integers. For example, an action could be represented as a single integer in the range ([0, 1048575]) (which is (32^4 - 1)). This integer can then be decoded into the 4 actions for the elements 7, 10, 13, and 16.
To encode and decode actions, you can use base-32 representation since you have 32 possible states for each action.
Decoding Actions in the Step Function:
When you receive an action in your "StepFunctTest", it will be a single integer. You need to decode this integer into 4 separate actions. Here's how you could do it:
function [NextObservation, Reward, IsDone, UpdatedInfo] = StepFunctTest(Action, Info)
% Decode the single action into 4 separate actions
actions = zeros(1, 4); % Initialize array to hold decoded actions
for i = 1:4
actions(i) = mod(Action, 32); % Get remainder (current action)
Action = floor(Action / 32); % Reduce Action for the next iteration
end
% Now, actions(1), actions(2), actions(3), and actions(4) represent the actions
% for elements 7, 10, 13, and 16 respectively
% Apply actions
% Assuming you have a way to map the 0-31 action to your desired binary format
% For simplicity, assuming actionMap is a function that maps the action index to its binary representation
DSSText.command = sprintf('New LoadShape.estado_cap_1 npts=24 interval=1 mult=(0 0 0 0 0 0 %d 0 0 0 %d 0 0 0 %d 0 0 0 %d 0 0 0 0 0)', actionMap(actions(1)), actionMap(actions(2)), actionMap(actions(3)), actionMap(actions(4)));
% Continue with the rest of your step function
end
function binaryAction = actionMap(index)
% Here you would map the index (0-31) to its corresponding binary action
% This is a placeholder function. You need to implement the mapping based on your specific needs
binaryAction = [0 0 0 0 0]; % Example placeholder return value
end
This workaround allows you to extend your RL environment to handle actions on multiple elements without explicitly defining every possible combination.
Hope this helps!
