Preallocating for speed with increments

1 view (last 30 days)
I am trying to improve the efficiency of my code (the code I am running now takes several minutes to execute) and parts of my code go through a for loop using increments (e.g. 1:2:100).
Currently, the code looks as follows:
COL = 196:200;
f = F(:,COL)*ones(5,1);
D = 0;
for j = 1:35
B = 0;
for k = j:35:1435
A = f(k,:);
B = [B; A];
end
C = ones(1,42)*B;
D = [D; C];
end
fUS = D(2:36,:);
With F being a 1435x205 matrix.
The end result fUS, is a 35x1 vector.
At the part "for k = j:35:1435"
I would like to use something like A(k,:) = f(k,:) so that I can preallocate A for speed and then drop B.
However, the length of k is 41 while it increases from 1 to 36 to 71 to 106 and so on. The code above would put f(k,:) in the 1st, 36th, 71th column and so on, while I really want it to be in the 1st, 2nd, 3rd column and so on. How can I make it so that in each itteration of k it will put the value below the value in the previous itteration like it does in
for k = j:35:1435
A = f(k,:);
B = [B; A];
end
I am kind of lost.

Accepted Answer

Stephen23
Stephen23 on 22 Mar 2022
Edited: Stephen23 on 22 Mar 2022
Indexing would be a much better approach, rather than those inefficient low-level loops. Note that it is not that loops are generally inefficient, just those particular loops which expand arrays on every iteration. Avoid doing that.
For example, that entire inner loop can be replaced with this much simpler and more efficient code using very basic indexing:
C = sum(f(j:35:1435,:))
However your entire code can be replaced with this:
fUS = sum(reshape(f,35,[]),2)
  2 Comments
Hylke Dijkstra
Hylke Dijkstra on 22 Mar 2022
That looks a lot better. Would you have any advice or handholding on how to interpret it? Or anything that I could read up on to understand it better?
Thanks
Stephen23
Stephen23 on 23 Mar 2022
Edited: Stephen23 on 23 Mar 2022
"Would you have any advice or handholding on how to interpret it?"
It is not clear to me what you want to "interpret".
If you want to kow how to approach solving problems with MATLAB, then this might help:
"Or anything that I could read up on to understand it better?"
How to understand MATLAB:
How to understand the code I showed you: read the documentation for RESHAPE and SUM.

Sign in to comment.

More Answers (1)

Hylke Dijkstra
Hylke Dijkstra on 23 Mar 2022
YUS = Y(:,ROW);
B = zeros(1,35);
C = B;
for h = 1:35
for l = h:35:1435
A = YUS(l,:);
B = [B; A];
end
B = ones(1,42)*B;
C = [C; B];
B = zeros(1,35);
end
YUS = C(2:36,:);
YUS is a 1435x35 matrix. I am trying to figure out how to write this more efficiently, but I can't seem to figure out where to start. Does anyone know some hints on where to get started using reshape commands instead?
  3 Comments
Hylke Dijkstra
Hylke Dijkstra on 25 Mar 2022
Edited: Hylke Dijkstra on 25 Mar 2022
Thank you so much!
I will try to wrap my head around it.
I do not perfectly understand what you mean by ''the order of the element in memory or the linear index order".
Stephen23
Stephen23 on 25 Mar 2022
Edited: Stephen23 on 25 Mar 2022
''the order of the element in memory" means the order of the array elements as they are stored in your computer's memory. Computers do not have memory that stores arrays of arbitrary size and dimensions as an array in some kind of array-shaped memory, instead all arrays are stored as a long 1D list of values together with some meta-information about the size of the array. Consider this simple matrix with size 2x3:
M = [11,12,13;21,22,23]
M = 2×3
11 12 13 21 22 23
Is it somehow stored in your computer's memory exactly like we see it on the screen, as a matrix? No, it is not.
It is stored just like every other array, as a 1D list of values together with meta-information giving the size. The important question: what is the correspondence between that array and the list? There are two obvious answers:
  • column major: [11,21,12,22,13,23] (MATLAB, FORTRAN, R, Julia, mathematics)
  • row major: [11,12,13,21,22,23] (C, C++, ...)
What you see are the two ways that those matrix elements will be actually stored in memory. Every app that works with arrays will store the values using one of those two orders. MATLAB uses column-major. It is the order used with linear indexing (one of MATLAB's three powerful types of indexing) or when using RESHAPE, or anything else that refers to all elements of an array. (As an aside: note that the column-major order exactly matches the order of the dimensions of a matrix, whereas confused row-major computer programmers get the first two dimensions mixed up. That correspondence is even more obvious/important when you consider higher-dimension arrays).
Lets try linear indexing into our matrix:
V = M(:)
V = 6×1
11 21 12 22 13 23
And look at that: the order in V is the order of those elements stored in memory. Exactly the same concept applies to RESHAPE: it obtains the array elements in the order that they are stored in memory. Which explains why we get this ouput:
W = reshape(M,3,2)
W = 3×2
11 22 21 13 12 23
Note that M, V, and W all have the elements in exactly the same order: at no point has the element order changed. The only thing that has really changed in some meta-information about the array size (which is why RESHAPE is very fast, should you ever be interested in writing fast, efficient code).
That is why the most important thing you need to keep in mind is the order of the elements in memory.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!