How to save only latest ~10 iterations on for loop

I'm trying to run a for loop that will give me a new iteration of a matrix at each time step. The problem comes in because I need to calculate up to 20,000 time steps, but I don't have any where near the memory for that. I'm only interesting in the last ~ 10 time steps. Is there some way to write over or erase the older time steps?

Answer 1

[Geoff Hayes]

If you are only interested in the latest 10 matrices, then you can create a "circular" cell array of fixed size that is updated at each iteration, overwriting the older matrices as the code iterates:

 circularArraySz = 10;
 circularArray   = cell(circularArraySz,1); % create the circular cell array
 nextIdx         = 1;                       % the index into the array in which
                                            % to insert the next matrix

 % do the for loop
 for i=1:20000

     % do the work to generate the matrix A

     % add A to the cell array
     circularArray{nextIdx} = A;

     % increment to the next index
     nextIdx = nextIdx + 1;

     % wrap around to the beginning of the circular array if the index
     % is greater than circularArraySz
     if nextIdx>circularArraySz
         nextIdx = 1;
     end
 end

When the loop ends, circularArray will have the last ten matrices. Try it out and see what happens!

[sasha]

Just to clarify this line:

 circularArray   = cell(circularArraySz)

would be where I preallocate the size?

At the moment mine currently has:

A= zeros(Nx,Ny,20000)

So I would get rid of that line and replace by what you wrote?

[Geoff Hayes]

Yes - though I made a mistake in my previous post. That line should read

 circularArray   = cell(circularArraySz,1);

so that it is a 10x1 cell array (and not 10x10). And it could be used to replace your A matrix.

Alternatively, since it appears that your matrix is the same size on each iteration, you could replace the circular cell array with something more like what you have

 circularArray = zeros(Nx,Ny,10);

and then just update in the same way on each iteration with nextIdx

 A(:,:,nextIdx) = mtxForThisIteation;

[sasha]

I just tried this, and ran into a problem. The value of A is actually a function of the iteration it's on. Or rather:

for n=1:20000
   f(x,y,n+1) = f(x,y,n) + g(n)
end

Where I've got f(x,y,1) and g(n) defined already. So when I loop back to

nextIndx=1

I start back at the beginning and I don't get any of the values of the function at later times.

Any ideas?

[Geoff Hayes]

Okay - so you just need an additional "pointer" or index to the previously inserted element into the circular array

 circularArraySz      = 10;
 circularArray        = zeros(Nx,Ny,circularArraySz);
 circularArray(:,:,1) = f(x,y,1);
 nextIdx              = 2;   
 prevIdx              = 1;

 for n=1:20000

     % do the calculation
     circularArray(:,:,nextIdx) = circularArray(:,:,prevIdx) + g(n);

     % save the index of the most recent addition to the array
     prevIdx = nextIdx;

     % increment to the next index as before
 end

[sasha]

Thanks! This works for my test functions!

[Geoff Hayes]

Awesome!

Answer 2

[Azzi Abdelmalek]

look at this example, I'm not sur if it's the best way

a=[];
for k=1:20
  a(end+1)=k
  a(1:end-10)=[]
end