Out of memory when removing column of a large matrix

Hi there,

When I want to remove column X of a 5343x11177 matrix called M by the command M(:,X) = [] I get an Out of Memory error. I have cleared all unnecessary matrices and vectors and still got the problem. Does anyone know how to fix this? Or is there another way to remove that specific column?

Result of whos M:

 whos M
  Name         Size                   Bytes  Class     Attributes

M         5343x11177            477749688  double

[Anneke]

Maybe this does matter too: There are still two large vectors that I need later, a 5343x1 double vector and a 11177x1 cell array, so I still have them in my workspace..

Answer 1

[Walter Roberson]

When you remove information from a matrix, MATLAB makes a temporary copy of the matrix. It is not able to do the replacement "in place", even if it is only trailing columns that are being removed. There might be a MATLAB File Exchange contribution that can remove columns "in place".

Answer 2

[Anneke]

I think I might have found the solution to my problem. Because M is a matrix with a lot of zero entries, I switched to M = sparse(M) and when I want to remove a column with the same command, it works!

Result of whos M:

 whos M
  Name         Size                Bytes  Class     Attributes

M         5343x11177            456984  double    sparse

As you can see below the amount of bytes only 0.1% of the original size. But I'm still curious how it would work if your matrix is not sparse..

Answer 3

[Titus Edelhofer]

Hi,

with this sparsity: yes, using sparse matrices is indeed the solution. For the original question: no, there is no "real" solution. You might have workarounds (e.g., don't really delete the column but have some index

idx = 1:size(M,2);
idx(X) = [];

and then work with the index idx for the columns. But whatever you would want to to, you would always have to use small parts of the matrix, because one copy of the matrix does not fit into your memory, as you saw...

Probably one could write a mex file copying the memory of the columns X+1:end to X:end-1, without reallocating memory and setting sizes but to call this a "dirty trick" would be a nice wording for doing this.

Titus