script running for 13 hours. should I be worried?

2 views (last 30 days)
I have a function going through 9 excel files with about 1.7mil cells. It imports them, puts everything into one column then goes through and removes all the 0's and saves the output of about 300k cells. Its been running for about 13 hours of so now and I'm concerned that I may have messed up the code so I was hoping someone would help me out a fresh set of eyes.
it does look like its working, maybe? the output files look with theyve been edited, but I can't really be sure. this is the code i'm using at the moment.
function [ m ] = combine( input_args )
source = 'C:\Users\aoneil\Desktop\ODEN\trimmed'; %input the directory to choose the source folder
dest_dir = 'C:\Users\aoneil\Desktop\ODEN\graphs'; %choose where to output the resulting file
files = dir(fullfile(source, '*.CSV'));
for j = 1:length(files) %loops through all the file in the directory folder
a = xlsread(fullfile(source, files(j).name));
for x = 1 : size(a,1) %steps through the matrix
d((1+((x-1)*size(a,2))):(size(a,2)*x), 1) = a(x,1:end); %saves all the values of original to matrix "d"
end
n=1;
for y=1:size(d,1)
if d(y,1) > 0
m(n,1) = d(y,1)
n=n+1;
end
end
csvwrite(fullfile(dest_dir, files(j).name), m); %outputs matrix "d" to the file name/directory chosen
end
end

Accepted Answer

Jan
Jan on 5 Feb 2015
You urgently require a "pre-allocation". Letting arrays grow iteratively is extremely expensive. Example:
x = [];
for k = 1:1e5
x[k] = rand;
end
Now in each iteration a completely new vector x is created and the older values are copied. This means, that the OS has to reserve sum(1:1e5) * 8 bytes in total and copy almost the same amount of data in addition.
A vectorization solves the pre-allocation implicitly:
n=1;
for y=1:size(d,1)
if d(y,1) > 0
m(n,1) = d(y,1)
n=n+1;
end
end
Better without loop and IF:
m = d(d > 0);

More Answers (1)

Alessandro Masullo
Alessandro Masullo on 5 Feb 2015
You can use a task manager to check if matlab is still using the CPU. If not, it probably got stuck.

Categories

Find more on Data Import from MATLAB in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!