How to solve Matlab memory issue

Hi all

I am working with huge data file of matrix size 1024*736 and has 367 such files. The approximate RAM required for processing this is 16 GB. I am using Matlab 64-bit version in Linux of 16 GB capacity. But matlab freezes even at 4 GB. What is causing the problem? Is Matlab is not configured to utilise the system memory at all. Are the matlab and system are not working in hands to solve memory problems. How to get rid of this?

Thanks for all your support.

[Walter Roberson]

Are you sure about the processing memory? If the matrices are double precision, then the total memory of what you outline is a bit over 2 Gb to load them. What kind of operation do you do on them after they are loaded?

[Kavitha Srinivasan]

Yes. the matrices are stored as double and I used FDk reconstruction operation, (that involves weighting, filtering, back projection) of those input files.

Answer 1

[Jan]

What is the output of:

[a,b,c] = computer

The memopry can be exhausted by other problems also, e.g. when a pre-allocation is forgotten. Then the finals amount of memory might be some GB only, but temporarily thousands of GB could be allocated. So please post a minimal example which reproduces the problem. Usually rand is sufficient to simulate reading large files.

[Kavitha Srinivasan]

Hi

The output of [a,b,c] = computer is:

a= GLNXA64; b= 2.8147e+14;c='L'

The script file is too big and it basically builds GUI and through which the input files are loaded and processed. The operation involved is FDK reconstruction algorithm that includes weighting of projections, filtering and then back projection. the app RAM required for processing this is calculated by this function and displayed on GUI as below.

function memstr=OSCaRGUIMemory(nx,ny,nz,nu,nv)
%Notes:
% 1KB = 1024B.
% 1MB = 1048576B.
% 1GB = 1073741824B.

memo = 8*(4*nx*ny*nz + 2*nu*nv);
if memo>1023 && memo< 1048576
        units='KB';
        memo=memo/1024;
else
    if memo> 1048575 && memo< 1073741824
        units=' MB';
        memo=memo/1048576;
    else
        if memo>1073741823
            units=' GB';
            memo=memo/1073741824;
        else
            units=' Bytes';
        end
    end        
end
memstr=strcat(num2str(2^nextpow2(memo)),units);

Thanks

[Walter Roberson]

(FDK appears to mean Feldkamp-Davis-Kress)

[Kavitha Srinivasan]

yes. its cone beam reconstruction algorithm

Answer 2

[Matt J]

To do FDK reconstruction, you don't have to load the entire data set into memory at the same time. Traditionally, it is done by processing one projection view at a time. If your 367 files each contain a projection view, why not just load one of them into RAM at a time, do the processing needed to update your image volume, ad then throw it away?

Also, how do you rationalize

memo = 8*(4*nx*ny*nz + 2*nu*nv);

If your image volume is being held as 4-byte singles, shouldn't it just be

memo = 4*nx*ny*nz;