Parfor vs For: Different Solutions

Hi, estimate function comes up with different results when we use it in for and parfor loops with ARIMA models having large parameters such as AR> 6 and MA>5. We use fmincon and change the tolerances (made it the same in each cases), and then observe that depending on the tolerance values, the results may coincide (but not in all cases). In order to understand if for or parfor gives the " correct" (more reliable) result, we give the ARIMA models to Economic Modeler. The results of the modeler are the same with results that we find by using for loop with default parameters. When we have examined each iterations of fmincon (used in estimate function), the algorithms do not stop at the same iteration even though the tolerance values have been the same (when either default or given by us). In parfor case, the algorithm iterates more and comes up with a result having higher objective value, namely, maximum likelihood, where it is a minimization problem.

In short: we would like to understand what causes that parfor and for loops come up with different results and how we can be sure that they give the same results.

Answer 1

[Walter Roberson]

Unless you are using a relatively recent MATLAB release and have made specific configuration changes, then each parfor worker will only have access to one core.

When vectorized operations are done on large arrays, LAPACK or Intel MKL will be invoked to automatically run on all available cores. The work is divided up by the number of available cores, partial work is done, and then the results are merged. When only one core is available that could be the equivalent of doing the operation in serial, whereas with regular for with multiple cores available the operation might have been split between them. This can make a difference because floating point operations are not transitive. In floating point, A+B+C is not generally equal to A+(B+C) . If you have a vector, then

 T = 0; for K = 1 : numel(A); T = T + A(K); end

will not generally have the same result as

N4 = numel(A)/4;
T = zeros(1,4);
for K = 1 : numel(A);
  T(1) = T(1) + A(K);
  T(2) = T(2) + A(K+N4);
  T(3) = T(3) + A(K+2*N4);
  T(4) = T(4) + A(K+3*N4);
end
T = T(1) + T(2) + T(3) + T(4);

which is equivalent to what might happen if sum(A) is split between 4 cores.

http://matlab.wikia.com/wiki/FAQ#Why_is_0.3_-_0.2_-_0.1_.28or_similar.29_not_equal_to_zero.3F

These small differences caused by different order of operations can accumulate. With non-linear equations, the small changes can become important.

In the particular case of sum(), you can improve accuracy using Jan's Compensated Sum