cell2mat loops & running out of memory

6 views (last 30 days)
Aloha,
I need to process 72 hours of sampling at a rate of 25.6 kHz. The MatLab files I am given have the form of a structure, with string fields containing instrument information, units, etc. The numerical data itself is contained in 262144x2 double fields named StreamSegment000000_000255, StreamSegment000256_000511, and so forth, each of which containing 10.24 seconds of data. The file I am currently working on has roughly 2 hours of data, divided into 703 10.24-second fields of size 262144x2 double and a 2.84-second field of size 72704x2 double (of course, files will have different sizes). The first thing I need to compute is 30-second RMS for all 72 hours. Here is my approach:
S=load('REC0049.mat');
cell0=struct2cell(S);
C=cell0(3:end,:); % crops out the first 2 arrays of junk information
Now cell2mat(C) will of course lead me "??? Error using ==> cat; Out of memory", so I tried to break down to several matrices.
for i=1:number
eval(sprintf('M%d = cell2mat(C((1+3*(i-1)):(3*i),:));', i));
end
If number=235, obtaining 235 matrices each containing 3 of these 10.24-second fields would allow me to easily compute 30.72-second rms (close enough) for these 2 hours. Of course, this works just fine for small "numbers", but I quickly encounter "*??? Error using ==> cat Out of memory" again.
I could try nested for-loops and progressively clear out the memory, but I am sure there are solutions that are a lot more elegant and efficient.
Thank you!

Accepted Answer

Matt J
Matt J on 4 Jan 2013
Edited: Matt J on 4 Jan 2013
So if I understood you, the fields named StreamSegment000000_000255, StreamSegment000256_000511 are the fields of S when you do
S=load('REC0049.mat');
In other words, you generated REC0049.mat by using eval to auto-name 703 different variables and saved them to a .mat file? I hope you know that that's a bad idea, as described here.
Instead, you could use MATFILE to write the segments to and from the elements of a cell array C in REC0049.mat.
C=cell(1,number);
save REC0049 C;
matObj = matfile('REC0049.mat','Writable',true);
for i=1:number*3
matObj.C(1,i)={data_chunk_to_write};
end
Then later
rms=zeros(1,number);
for i=1:number
rms(i) = computeRMS( cell2mat(matObj.C(1, (1+3*(i-1)):(3*i),:) ) )
end
  1 Comment
Margaux
Margaux on 4 Jan 2013
Thanks. I did not create the REC0049.mat file. The fields named StreamSegment000000_000255, StreamSegment000256_000511 etc. were generated when I loaded the file (without the S= statement).

Sign in to comment.

More Answers (1)

Malcolm Lidierth
Malcolm Lidierth on 5 Jan 2013
Depending on the MAT-file version, there may be something that could help in:

Categories

Find more on Cell Arrays in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!