Calculating the entropy of a discrete memoryless source?
4 views (last 30 days)
Show older comments
How could I calculate the entropy of a discrete memoryless source S={a,b,c} in unit of bits? What would be the matlab script for calculating this?
0 Comments
Answers (3)
Vinod Sudheesh
on 27 May 2015
Hello Daniel,
You could write the script for the same as follows
1) Create a function, say "esingle" that computes the "entropy" of a symbol in the source alphabet. The definition of the function "esingle" could be as follows
function Y=esingle(p)
if(p==0)
Y=0;
return;
else
Y=p*log2(1/p);
end
end
2) Now invoke this function "esingle" using the syntax below to calculate the "entropy" of the source alphabet "a".
a=[0 1 1];
E=sum(arrayfun(@esingle,a/sum(a)));
Hope this helps!
Thanks Vinod
Walter Roberson
on 27 May 2015
A source S={a,b,c} is a source with three alphabetic symbols of no particular numeric value. If we know the generating equations for S then we analyze it analytically to determine the entropy. Otherwise the best we can do is estimate the entropy from a stream of the generated symbols. If we have assigned definite and distinct numeric values to "a", "b", and "c" and if "s" is a finite stream of those exact values, then
counts = histc(s, sort([a,b,c]));
probs = counts ./ length(s);
nzprobs = probs(probs>0);
entropy = sum(nzprobs .* log2(1./nzprobs));
Note: this code will work even if the assigned values are characters and s is a string.
0 Comments
rakesh ranjan
on 17 Nov 2020
function Y=esingle(p)
if(p==0)
Y=0;
return;
else
Y=p*log2(1/p);
end
end
0 Comments
See Also
Categories
Find more on QAM in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!