Error 'The process cannot access the file because it is being used by another process' during a parfor loop

I am using parfor loop and system() function to run batches of simulations based on a codgenerated file from a simulink model, and writing to output files and logs.

And on Win10, I get an error like the following:

Error using driftTrans_rsim (line 129)
Error running batch file 'driftTrans_001' :
C:\github\DriftControl\rapid_sim_Win10>echo "driftTrans.exe -p driftTrans_prm_sets.mat@1 -o driftTrans_run1.mat -L 60 2>&1>> driftTrans_001_run_scr.log"
"driftTrans.exe -p driftTrans_prm_sets.mat@1 -o driftTrans_run1.mat -L 60 2>&1>> driftTrans_001_run_scr.log"

C:\github\DriftControl\rapid_sim_Win10>driftTrans.exe -p driftTrans_prm_sets.mat@1 -o driftTrans_run1.mat -L 60  2>&1 1>>driftTrans_001_run_scr.log
The process cannot access the file because it is being used by another process.

C:\github\DriftControl\rapid_sim_Win10>echo "driftTrans.exe -p driftTrans_prm_sets.mat@2 -o driftTrans_run2.mat -L 60 2>&1>> driftTrans_001_run_scr.log"
"driftTrans.exe -p driftTrans_prm_sets.mat@2 -o driftTrans_run2.mat -L 60 2>&1>> driftTrans_001_run_scr.log"

C:\github\DriftControl\rapid_sim_Win10>driftTrans.exe -p driftTrans_prm_sets.mat@2 -o driftTrans_run2.mat -L 60  2>&1 1>>driftTrans_001_run_scr.log
The process cannot access the file because it is being used by another process.

It seems that the parallel workers have conflict in accessing files. There are four types of files involved:

1 model file, with the name driftTrans.exe;

2 parameter sets file, with the name driftTrans_prm_sets.mat;

3 simulation output file, with the names driftTrans_run1.mat, driftTrans_run2.mat, ...;

4 log file, with the names driftTrans_001_run_scr.log, driftTrans_002_run_scr.log, ....

While generating the batch script, I made sure that each line creates a unique simulation output file name; however, the log files are per batch file, and every batch file contains 20 lines to run 20 simulations. What I cannot tell is, whether the error was because of writing to the same log file, or because of multiple workers trying to access the same model file driftTrans.exe.

Any tip on how to identify the source of the error? Also just a note, the program runs perfectly fine on an Ubuntu 18.04.6 OS, but causes error in Windows 10.

Answer 1

[Walter Roberson]

Your code is using the same error log file for both. driftTrans_001_run_scr.log . This is unlike your text description where you expect driftTrans_002_run_scr.log for the second one.

[Tong Zhao]

Thanks Walter. I am trying to now remove the logging, but I do not know how to modify the batch commands to do so. The batch commands I have is like the following

driftTrans.exe -p driftTrans_prm_sets.mat@1 -o driftTrans_run1.mat -L 60 2>&1>> driftTrans_001_run_scr.log

Do you know how to remove the logging process? I'll give it a go after the modification.