Timestep problems using Reinforcement Learning with EnergyPlus

I am currently trying to train an agent using an EnergyPlus Model.

Herefore I set up a Simulink Environment with observation, reward, agent and EnergyPlus Simulation block. The model and the EnergyPlus Simulation starts to run but then suddenly stops with error messages (code below). Does anyone have an idea how to solve this issue or where I should look for to find a solution?

Another question I have is regarding the agent and EnergyPlus. I want the agent to give me a value between 0 and 1. I want to use this value to simulate 1 hour in EnergyPlus. The agent should not output any values until I have the results(observations) of the first simulated hour. How do I have to set this up?

Error using rl.env.AbstractEnv/simWithPolicy (line 70)
An error occurred while simulating "KleinesModellSimulinka" with the agent "agent".

Error in rl.task.SeriesTrainTask/runImpl (line 33)
            [varargout{1},varargout{2}] = simWithPolicy(this.Env,this.Agent,simOpts);

Error in rl.task.Task/run (line 21)
            [varargout{1:nargout}] = runImpl(this);

Error in rl.task.TaskSpec/internal_run (line 159)
            [varargout{1:nargout}] = run(task);

Error in rl.task.TaskSpec/runDirect (line 163)
            [this.Outputs{1:getNumOutputs(this)}] = internal_run(this);

Error in rl.task.TaskSpec/runScalarTask (line 187)
                runDirect(this);

Error in rl.task.TaskSpec/run (line 69)
                runScalarTask(task);

Error in rl.train.SeriesTrainer/run (line 24)
            run(seriestaskspec);

Error in rl.train.TrainingManager/train (line 291)
            run(trainer);

Error in rl.train.TrainingManager/run (line 160)
            train(this);

Error in rl.agent.AbstractAgent/train (line 54)
TrainingStatistics = run(trainMgr);

Error in KleinesModell (line 90)
    trainingStats = train(agent,env,trainingOptions);

Caused by:
    Error using rl.env.SimulinkEnvWithAgent>localHandleSimoutErrors (line 689)
    You cannot set the read-only property 'timestep' of mlep.

[Rezky Nanda]

Hi,

I'm currently having the same problem. Have you found the solution to the problem yet?

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

I am not familiar with EnergyPlus so I am not sure about the error you are getting (although it seems like you are trying to change the 'timestep' property?), but if you want the agent to provide actions every hour, you just need to set the Ts/sample time agent option to this value.

Answer 2

[Byung - Gi Jeon]

can i see your code and Energyplus file?

Please contact me via my email if possible. ([e-mail])