Variables Unrecognized in parfor but Work in for Loop
Show older comments
I have a struct variable with which I extract each variable using 'eval()'. I then call those variables in a parfor loop and I recieve an error (provided below). Notably, if I define the variable explicitly and don't use 'eval()', the parfor loop works fine... Why is this the case and are there any fixes? I would rather not extract each variable from the struct manually in my code because there are many variables and it would make the code look unorderly.
I would appreciate any suggestions. Thank you!
"Error using *FUNCTION NAME* (line 26)
The source code (*FILE PATH*) for the
parfor-loop that is trying to execute on the worker could not be found.
Caused by:
Unrecognized function or variable 'x0'.
Worker unable to find file.
Unrecognized function or variable 'x0'."
The simplified code is provided as follows:
function [output] = *FUNCTION NAME*(params)
% Start Parallel processing
parpool('local',8)
% Extract variables from the struct
fields = fieldnames(params);
for i = 1:length(fields)
eval([fields{i} ' = params.' fields{i} ';']); % Extract variable from struct (i.e. attain x0)
end
parfor j = 1:NumTrials
[x, z] = truth_gen(x0);
*ADDITIONAL CODE*
end
*ADDITIONAL CODE*
end
EDIT: Upon a more rigorous search, it appears that using 'eval' is bad practice! Instead I will save my variables to a mat file and reload them: https://www.mathworks.com/matlabcentral/answers/299135-is-it-possible-to-extract-all-fields-from-a-structure-automatically#answer_231447
However, for my own knowledge, I am still curious why the creation of variables (via 'eval') does not work in parfor.
EDIT 2: It seems saving the variables in a mat file and loading them still causes the same error. Can someone explain why?
Accepted Answer
You should not be using eval to create new variables, regardless of whether the loop is parfor or a regular for loop, see also,
Use dot-indexing expressions params.fieldname to reference the variables. If you prefer numeric indices instead of field names, move the data to a different container, like a cell array, e.g.,
params.a=1:3; params.b=4:5;
s=struct2cell(params),
9 Comments
However, for my own knowledge, I am still curious why the creation of variables (via 'eval') does not work in parfor.
If you use eval instead of explicit code to create variables, the parfor parser cannot see how the variable was created, which means it cannot know how to classify the variables into the necessary categories decribed here,
Aneesh
on 2 Apr 2025
Matt J
on 2 Apr 2025
I wish there was a way to notice this in my workspace/command window when I was debugging.
For me, your code produces the Code Analyzer warning below.
Aneesh
on 2 Apr 2025
Edric Ellis
on 2 Apr 2025
Matt's explanation is exactly correct. When you create variables using eval ahead of the parfor-loop, those variables are not present in the text of your program. To ensure efficient operation, parfor sends to the workers only those variables it knows for sure are used in the body of the loop. It works this out using static analysis. So, because there is no statement inside your function assigning a value directly to x0, parfor assumes that it does not need to send that variable into the body of the loop. (Further, it assumes that x0 will turn out to be a function).
Any means of dynamically creating variables without explicit assignment statements to those variables (i.e. using load without an output argument, or assignin, or eval, or evalin, ...) will hit the same restriction.
This section of the documentation https://uk.mathworks.com/help/parallel-computing/troubleshoot-variables-in-parfor-loops.html#bvlk718-1 is referring to this limitation.
Mohammad Sami
on 2 Apr 2025
Is there a reason why you can't directly reference the params variable as follows ?
[x, z] = truth_gen(params.x0);
Matt, the eval is NOT within the parfor in the code I provided, so that is not the issue.
Yes, I see now that you do not get a warning, but that is one of the reasons eval is discouraged. The Code Analyzer's ability to analyze what eval is doing is limited.
EDIT 2: It seems saving the variables in a mat file and loading them still causes the same error. Can someone explain why?
It depends how you are using load(). If you are returning an output from load(), there should be no problem,
params=load(___)
Otherwise, you are creating variables non-explicitly in the same manner as eval() and run into the same problem. For that reason, using load() without an output argument is generally discouraged, for the same reasons as eval is discouraged.
Aneesh
on 2 Apr 2025
"I did not know that by doing 'eval' or 'load' I was dynamically creating variables, I simply thought this was a normal operation of assigning variables in my workspace."
For an interpreted language there might not be much difference. But because the MATLAB Engine uses static code analysis to optimize and compile code before it is run there is a very big difference between them:
Performance is only one reason to avoid dynamically creating/accessing variables:
More Answers (1)
I am aware of this, but I was hoping to avoid writing 'params.' to reduce clutter in my code.
You might consider this File Exchange download,
It doesn't avoid the need to explicitly unpack the struct, but it reduces the manual effort of writing code to do so, and can also automatically layout the statements in a way that condenses the code. Example, suppose you have a 30-field struct,
args=cell(2,30);
args(1,:)=cellstr("x"+(1:30));
p=struct(args{:}) %struct with 30 fields
Then you can auto-generate the unpacking statements by doing as below and copy/pasting the output text into your code.
columns=6;
structvars(columns,p) %copy-paste the output into your code
Categories
Find more on Parallel for-Loops (parfor) in Help Center and File Exchange
