# large file slow running

2 views (last 30 days)
Joydeb Saha on 20 Sep 2021
Answered: Jan on 20 Sep 2021
latmax=90;
longmin=-180; a=1;
longmax=180;
data=zeros(90,90);
for b=1:length(x(:,1))
if (x(b,8)>=longmin && x(b,8)<longmax && x(b,7)>=latmin && x(b,7)<latmax);
for col=1:13
data(a,col)=x(b,col);
end
a=a+1;
end
end
theres a file called x.mat (601760x13). What might be the reason it runs too slow
KSSV on 20 Sep 2021
What you are trying to achieve? It can be achived without loop. You need to give an explanation of your problem. What does x.mat have?

Jan on 20 Sep 2021
I guess that the iterative growing of the array data slows down the processing.
Remember that the growing of an array required to create a new array and to copy the foermer contents.
x = [];
for k = 1:1e6
x(k) = rand;
end
This creates a new array in each iteration. Although the final array needs 8MB only (8 bytes per double), Matlab has to allocate and to copy sum(1:1e6)*8 bytes, which are more than 4 TB ! The solution is easy: Allocate with the final size:
x = zeros(1, 1e6); % Pre-allocation
for k = 1:1e6
x(k) = rand;
end
In your case you do not know the needed size in advance. You tried to pre-allocate with data=zeros(90,90), but this is not the final size. You could determin the final size:
latmin = -90;
latmax = 90;
longmin = -180;
longmax = 180;
index = (x(:,8) >= longmin & x(:, 8) < longmax & x(:, 7) >= latmin && x(:,7) < latmax);
data = zeros(sum(index), 13);
a = 1;
for b = 1:size(x, 1) % nicer and faster than: length(x(:,1))
if index(b)
for col = 1:13
data(a, col) = x(b, col);
end
a = a + 1;
end
end
But it is easier, nicer and faster to perform the copy directly without a loop:
latmin = -90;
latmax = 90;
longmin = -180;
longmax = 180;
index = (x(:,8) >= longmin & x(:, 8) < longmax & x(:, 7) >= latmin && x(:,7) < latmax);
data = x(index, 1:13);