You are now following this question
- You will see updates in your followed content feed.
- You may receive emails, depending on your communication preferences.
Reliably determine RAM consumption of MATLAB variables
22 views (last 30 days)
Show older comments
I am looking for a reliable way to determine how much RAM is devoted to Matlab variable data at a given moment. The whos() command does not work, because it doesn't account for copy-on-write sharing. For example, this code results in 2MB of Matlab data, but whos() counts it as 4MB,
A=rand(512);
B=A;
whos A B
Name Size Bytes Class Attributes
A 512x512 2097152 double
B 512x512 2097152 double
Likewise the memory() command is not reliable, because it requests a memory tally from the operating system, whose estimate can fluctuate from moment to moment depending on what the operating system is doing in the background.
There must be a way to do this, right?
14 Comments
Steven Lord
on 29 Feb 2024
I am looking for a reliable way to determine how much RAM is devoted to Matlab variable data at a given moment.
How would you plan to use that information were it available?
Matt J
on 29 Feb 2024
Edited: Matt J
on 29 Feb 2024
@Steven Lord Nothing beyond the obvious. It would allow me to measure how much my code, or sections of it, are consuming and thereby pursue more optimal memory usage.
John D'Errico
on 29 Feb 2024
Why must there be? In a system where the actual memory taken up by B there will change, and in a way that is not fully under your control, you don't want to rely on it being less. I'd just say if you are living that close to the edge, where the size of a variable matters, then you need to get more memeory, and at the same time, look for better ways to do what you are doing.
Matt J
on 29 Feb 2024
Edited: Matt J
on 29 Feb 2024
@John D'Errico Why would the memory taken up by B not be fully under my control? As the coder, I'm the one who manipulates B.
Matt J
on 29 Feb 2024
Edited: Matt J
on 29 Feb 2024
Perhaps as further motivation, I would point out that whos() doesn't even reliably report the RAM consumption of a single isolated variable. Below, whos() thinks that F consumes 4 MB, but we know from my initial post above that a 512x512 double matrix only consumes 2 MB (cf. also, this thread). Surely, the grid vectors and other incidental data in a griddedInterpolant object are second order in RAM consumption to the F.Values property.
F=griddedInterpolant(rand(512));
whos F
Name Size Bytes Class Attributes
F 1x1 4202736 griddedInterpolant
John D'Errico
on 29 Feb 2024
Edited: John D'Errico
on 29 Feb 2024
Yes, you do control it, sort of. I knew this would be a point of contention. At the same time though, MATLAB is the one making the decision, and you need to live with how it makes that choice.
My point is still, assume a variable takes up as much memory as it would take if fully fleshed out, and you will be safe. Second, to avoid living that close to the edge. You will fall off the cliff far too easily.
Walter Roberson
on 29 Feb 2024
The size required to represent griddedInterpolant varies with a complicated formula in the number of rows and columns. 4202736 is not twice 2097152.
Matt J
on 29 Feb 2024
Edited: Matt J
on 29 Feb 2024
The size required to represent griddedInterpolant varies with a complicated formula in the number of rows and columns. 4202736 is not twice 2097152.
It's pretty close:
4202736/2097152
ans = 2.0040
As I said, it is understood that there is other property data in a griddedInterpolant object that accounts for the difference from a perfect factor of two, but it is second order.
Here is another way to see that whos() is double-counting the memory consumption of F.Values for some reason. Below, the agreement between m1 and m2 clearly shows that V and F consume approximately the same memory, but whos() thinks it is twice that.
clear
M0=memory().MemUsedMATLAB/2^30; %baseline memory
V=rand(1e4);
m1=memory().MemUsedMATLAB/2^30 - M0 %consumption of V only
F=griddedInterpolant(V);
clear V;
m2=memory().MemUsedMATLAB/2^30 - M0 %consumption of F only
m3=whos('F').bytes/2^30 %consumption of F according to whos()
m1 =
0.7426
m2 =
0.7426
m3 =
1.4903
Matt J
on 29 Feb 2024
Edited: Matt J
on 29 Feb 2024
My point is still, assume a variable takes up as much memory as it would take if fully fleshed out, and you will be safe.
I feel like that's asking too much of the coder. The issue I am talking about applies beyond the simple example of two value-semantic copies that I presented. If A and B were handle objects (some people do a lot of work with those), I am supposed to be safe in the assumption that they will always share memory. And in a more serious example, I could have hundreds of handle-semantic copies all over the place. It would be way overconservative to have to pretend that N handle-semantic copies consume O(N) memory.
Steven Lord
on 29 Feb 2024
Since we're talking about objects, suppose I had a class with one public property and fifty private properties. Would you expect whos to show you the memory consumed by all fifty-one properties (thus exposing at least some information about those private implementation details) or would you expect it to just show the memory consumed by the one public property?
How about if this was a Java object? How much memory should MATLAB report?
P = java.lang.Double(pi)
P =
3.141592653589793
whos doesn't even try in this case.
whos P
Name Size Bytes Class Attributes
P 1x1 java.lang.Double
As other examples with some questions about your expected ideal behavior, since you referred to memory sharing:
clear all
A = ones(100);
B = A;
whos A B
Name Size Bytes Class Attributes
A 100x100 80000 double
B 100x100 80000 double
Would you expect this to show all the memory as belonging to A (with B listing 0 bytes), all the memory as belonging to B (with A listing 0 bytes), half to each, etc.? Why did you answer that way?
A = 0;
whos A B
Name Size Bytes Class Attributes
A 1x1 8 double
B 100x100 80000 double
If all the memory had belonged to A in the previous example, with B having been listed as having 0 unique bytes, would you be surprised by the sudden apparent "spike" in B's memory usage despite it not being changed or even referred to between the two whos calls? [If your previous question had listed B as owning all the memory, set B to 0 instead and use "B = A" in the next example.]
A = B;
whos A B
Name Size Bytes Class Attributes
A 100x100 80000 double
B 100x100 80000 double
Does A "reclaim its memory" (as listed in whos) from B, or is it now a copy of B and so consume 0 bytes? Why?
C = ones(100);
D = C;
C = D;
whos C D
Name Size Bytes Class Attributes
C 100x100 80000 double
D 100x100 80000 double
How much memory does C claim to own and how much memory does D claim? Should they claim "joint custody" and split the difference?
Matt J
on 29 Feb 2024
Edited: Matt J
on 29 Feb 2024
@Steven Lord I would be happy if the memory() command offered a total and accurate tally of all RAM consumed by variables, something like,
consumed = memory().MemUsedByMATLABVariables
It already tries to do this anyway, but because it uses the background OS, doesn't always get accurate or stable results.
Walter Roberson
on 29 Feb 2024
Edited: Walter Roberson
on 29 Feb 2024
for rectangular double matrices of size n by n, griddedInterpolant occupies 16*n*(n + 1) + 240 bytes
F = griddedInterpolant(rand(10));
struct(F)
Warning: Calling STRUCT on an object prevents the object from hiding its implementation details and should thus be avoided. Use DISP or DISPLAY to see the visible public details of an object. See 'help struct' for more information.
ans = struct with fields:
GridVectors: {[1 2 3 4 5 6 7 8 9 10] [1 2 3 4 5 6 7 8 9 10]}
Values: [10×10 double]
Method: 'linear'
ExtrapolationMethod: 'linear'
ValuesForCopy: [10×10 double]
UnderlyingInterpolantForCopy: []
I don't understand why ValuesForCopy exists, but it does...
The point is that griddedInterpolant "legitimately" takes up twice as much memory as the input array, and whos() does report its memory usage correctly.
Matt J
on 29 Feb 2024
Edited: Matt J
on 1 Mar 2024
@Walter Roberson I don't know if you're referring to a doc, but it still doesn't resolve the contradiction I presented to you in my previous post. According to your formula, m3 is correct, but that means memory() gave unreliable results. Which do we trust, and why?
Matt J
on 1 Mar 2024
Edited: Matt J
on 1 Mar 2024
@Walter Roberson the fact that ValuesForCopy exists doesn't prove that griddedInterpolant legitimately takes up twice as much memory. They could be shallow copies of one another. In fact, it seems the likely explanation for why memory() is a factor of 2 in disagreement with whos().
Answers (0)
See Also
Categories
Find more on Numeric Types in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!An Error Occurred
Unable to complete the action because of changes made to the page. Reload the page to see its updated state.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)