Is vectorization worth the trouble with recent MATLAB?

I started using MATLAB about 25 years ago, and at that time loops were multiples slower than doing vectorized calculations. Now it seems like there's not as much of a difference. Is vectorization still worth the trouble for a somewhat casual MATLAB programmer?

[Catalytic]

Whether it is "worth the trouble" depends on how much trouble it causes you. I think most Matlab programmers find vectorized coding easier than loops.

Answer 1

[Matt J]

Yes, it will always be worthwhile for a certain space of computations, because a plain for-loop will always be done sequentially, whereas vectorized operations are often internally parallelized.

A=rand(4000);

tic;
 c=0; for i=1:numel(A), c=c+A(i); end
toc

Elapsed time is 0.037629 seconds.

tic
  c=sum(A,'all');
toc

Elapsed time is 0.004962 seconds.

However, these days there are AI tools that will convert a loop to half-decent vectorized form for you, so you can probably draft your code in loop form much of the time, and then have it converted, if that's better for you.

[Mike]

Thanks. I thought I had read that MATLAB can essentially vectorize some loops before execution so there's not as much of a difference. Also I'm not running parallel MATLAB, so is the parallelization you describe still possible?

[Matt J]

If you mean you don't have the Parallel Computing Toolbox, then yes, the internal parallelization of built-in Matlab code is still done for you regardless.

[Walter Roberson]

There are a number of operations that for sufficiently large matrices Mathworks calls into higher-performance external libraries that are often multi-threaded. Sometimes the performance wins are just due to the high-performance code potentially being able to take advantage of SIMD (Single Instruction Multiple Data), but sometimes the task can be automatically split between multiple CPU cores. This sort of automatic parallelization is completely independent of Parallel Computing Toolbox. Indeed, by default Parallel Computing Toolbox restricts each worker to a single core, so by default Parallel Computing Toolbox workers do not get any benefit from automatic parallelization. (It is possible to configure Parallel Computing Toolbox workers to have access to multiple cores, at the cost of reducing the number of workers.)

Answer 2

[John D'Errico]

Is vectorization worth it? Let me see. For example: Add 1 to every element of a vector.

V = rand(1,1e6); % Which is itself highly vectorized!

% Unvectorized code:
for i = 1:numel(V)
  V(i) = V(i) + 1;
end

% vectorized code:
V = V + 1;

DUH. Surely you don't write code that uses a loop to do something as trivial? The latter form is vectorized, and I would at least hope you make use of such vectorizations all the time. In fact, MATLAB is designed to be a vectorized language. Why would you even seriously question the idea? So, yes. You DO want to use vectorized code whenever possible. This is what makes MATLAB easy to read, to write, to use, etc.

Anyway, I usually view vectorization as a way of getting MATLAB to use implicitly coded loops, so loops that operate under the covers, while the unvectorized coding has the loops viewable in the light of day. Some vectorization schemes trade off heavy use of memory to make the vectorization work. And that means some vectorization schemes may become unwieldy for large problems. It will be your skill and knowledge of the mathematics and the algorithm that will help you to decide which approach to take.

At the same time, vectorization is not always necessary. If I can envision a solution for a code that uses a loop, AND the loop is sufficiently short, AND I don't see an obviously vectorized solution, then I don't always waste programming time to find a vectorized solution.

This brings up an important point. Don't waste programming time, unless the code you can write is not sufficient to solve your problem in a reasonable time, as CPU time is cheap.

In short, yes, vectoriztion is still necessary. You would not be happy using MATLAB without it. But no, it is not always necessary. The dividing line falls for every programmer at different locations. The location of the line will depend upon your skill at MATLAB, your knowledge of mathematics, etc.

Answer 3

[Walter Roberson]

Let us take an example: finding the row indices of the first non-zero row in a 2D matrix.

A = full(sprand(10000,500,0.1));

Loop version:

tic
N = size(A,2);
idx1 = nan(N,1);
for K = 1 : N
    temp = find(A(:,K),1);
    if ~isempty(temp); idx1(K) = temp; end
end
toc

Elapsed time is 0.009280 seconds.

Vectorized version:

tic
temp = A == 0;
temp = cumprod(temp, 1);
idx2 = sum(temp,1).'+1;
idx2(idx2 == size(A,1)+1) = nan;
toc

Elapsed time is 0.015397 seconds.

compare

isequal(idx1, idx2)

ans = logical
   1

So looping is faster?

Do not be so quick to decide. Most runs of this exact same code, the vectorized version is roughly twice as fast. It depends on initial pattern of random numbers.

One thing that is for sure: the vectorized version takes significantly more memory, involving two temporary arrays the same size as the original data. If the array were sufficiently large, you might need to start paging memory to fit the temporary copies, and paging is very slow.

You cannot conclude much about whether for loops or vectorization is faster; we see from this that it varies with the data pattern.