Out of memory error

I know this is a commonly asked question but I've tried employing a number of suggested solutions and haven't managed to fix the problem so I'm hoping somebody can help me with this specific situation. I'm using a Dell XPS 13, windows 11. I have 16 GB of ram. I'm essentially trying to do some fourier transformations (time to frequency domain) on a big dataset of EEG data. 42 subjects, maximum of 32 channels (some channels removed due to noise depending on the subject) , 20 trials, 2048 time points. I'm not a fabulous coder so I could be missing some very easy tricks. I've tried pre-allocating all of the structs and arrays that I'm using, and removing any unecessary intermediate variables. I'd really rather not have to create a different way of organizing the data, but that will likely be my next try. I have a slightly more powerful computer I can use in my lab but I'd rather be able to do this on my personal computer. I've put my code below. It references the raw data variables (too large to attach here) 'alldat', and 'alllen'. 'alldat' is a 1x42 cell array. each cell contains 'singles' in the structure of channels x timepoints x trials, following the general pattern of 32x2048x20. 'alllen' is a double array of length 42 containing the number of channels for each subject. The first block of code is me trying to pre-allocate things to save memory, and it's run before the second block. I'm trying to get as much data as possible out of these functions so I'm very open to the idea that I'm just overloading my computer. Thank you in advance for your help!

%%
pow = cell([32 20]);
pow(:) = {NaN(1024,2047)};
snack(42).pow = [];
f = [];
t = [];
for l = 1:42
    snack(l).pow = pow;
end
%%
for sub = 1:42
    for trial = 1:20
        for ch = 1:alllen(sub)
            clear f t;
            [snack(sub).pow{ch,trial},f,t] = pspectrum(timetable(alldat{sub}(ch,:,trial)','SampleRate',256), 'spectrogram','OverlapPercent',99,'TimeResolution',0.01, 'FrequencyLimits',[1 100]);
        end
    end
end

Answer 1

[Jan]

32*20*1024*2047*42* 8 byte per element => Your array snack uses 451 GB RAM. If alldat contains additional data, the total amount of RAM to solve this problem should be 512 GB or more. Well, HUGE!

If the original data are single and not double, storing the result as singles might be sufficient. The occupy half of the memory: 4 bytes per element. But 225 GB is far beyond the available 16 GB also (ignoring the RAM required for OS and Matlab...).

Matlab can operate on "tall" arrays, which are larger than the memory. Then parts of the array are stored on the disk, and only the currently used parts are copied into the RAM. This is much slower than working in the RAM directly, of course. But this is no problem, if the processing can take some hours.

See: https://www.mathworks.com/help/matlab/tall-arrays.html

By the way, the pre-allocation of the elements of pow is a waste of time in your case: The cell elements are created with [1024x2047] elements, but inside the loop these arrays are simply overwritten. A pre-allocation is useful only, if elements are inserted in an existing array. So omit this line:

pow(:) = {NaN(1024,2047)};

Nevertheless, the final output still exceeds the available RAM.