huffmandict() The sum of elements of the probability vector must be 1

I am trying to use the huffmandict() function and it works if i use the sample program but gives me the error when i run this program

     clear all; close all; clc;

      load 'probMatrix.mat'
      symbols = {' ','0','1','2','3','4','5','6','7','8','9','A','B',...
           'C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S',...
           'T','U','V','W','X','Y','Z',};
      p = probMatrix;
     [dict,avglen] = huffmandict(symbols,p)
      samplecode = dict{5,2}

The probability matrix is calculated here.

Charz = ' 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ';
fi = fopen('testdata.txt');
inputData = fread(fi,'*char');
inputData = inputData';
totalCharz = length(inputData);
countMatrix = zeros(1,37);
probMatrix = zeros(1,37);

for count=1:length(Charz)
    x = strfind(inputData,Charz(count));
    charCount = length(x);

      % Saving Count
      countMatrix(1,count) = charCount;

end
% Finding Probabilities
probMatrix = countMatrix/totalCharz;

The sum(probMatrix) also gives the ans 1.0000 but i still get the error.

Error using huffmandict (line 107)
The sum of elements of the probability vector must be 1

Error in SampleHuffman (line 10)
[dict,avglen] = huffmandict(symbols,p)

Answer 1

[Roger Stafford]

I see two possibilities here. Either 1) there is some discrepancy between the 'totalCharz' value and the actual sum of 'countMatrix' values, which you should be able to check easily, or 2) the 'huffmandict' function is requiring an exact sum of 1 rather than allowing a tolerance for tiny round-off errors in the sum of probabilities.

If it is 1), the remedy is obvious: use sum(countMatrix) instead of totalCharz. Perhaps a character was encountered which is not one of your 37. If it is 2), and the difference between the actual sum and a perfect 1 is very tiny, make an adjustment to one or more of your probability values such as to produce an exact 1.

I will guess the problem is 1). It seems unreasonable to me that Mathworks would require an exact sum of 1 in 'huffmandict' without some allowance for round-off error.

[Amjad]

Well I have checked through the huffmandict.m and it is the roundoff error. it has a limit of something like 1*e-6 but the problem now is to how to reduce this error in the probMatrix that is how to increase the precision.

[Roger Stafford]

Well, if 'huffmandict' allows an error of 1*e-6, that should easily be satisfied by your code if 'totalCharz' were correct. Conclusion: The problem lies with 'totalCharz'. I would again strongly recommend you use sum(countMatrix) instead of 'totalCharz'. Don't be afraid. Try it!

[Amjad]

You were right. the length and sum were different.

length(inputData) =1096742

sum(countMatrix) = 1096739

I guess it was due to 2 unknown characters that were not in the string. Now i have converted to

totalCharz = sum(countMatrix)

and it works.

Thanks for the help