The covariance matrix in GMM go to singular. How can I fix it ? and Where is the problem ?

I am implementing GMM in matlab. But when I run that code, the covariance go to singular after 2 interations in EM-Algorithm. I don't know where is the problem. Data is 143-dimensional vector and about 300 observers. This is my code:
for (iter = 1:100)
fprintf(' EM Iteration %d\n', iter);
%%==========================a====================
%%STEP 3a: Expectation
% tinh lai trong so
pdf = zeros(m, k);
% For each cluster...
for (j = 1 : k)
% Tinh xac suat Gauss cho tat ca cac diem thuoc cluster 'j'.
pdf(:, j) = gaussianND(X, mu(j, :), sigma{j}); %<======this is my problem. Det(sigmal{j}) =0;
end
% pdf [m x k]
% phi [1 x k]
% pdf_w [m x k]
pdf_w = bsxfun(@times, pdf, phi);
% Divide the weighted probabilities by the sum of weighted probabilities for each cluster.
% sum(pdf_w, 2) -- sum over the clusters.
W = bsxfun(@rdivide, pdf_w, sum(pdf_w, 2));
%%===============================================
%%STEP 3b: Maximization
%%Calculate the probability for each data point for each distribution.
% store mean
prevMu = mu;
% Cap nhat cho moi cluster
for (j = 1 : k)
% update weight
phi(j) = mean(W(:, j), 1);
% update mean.
mu(j, :) = weightedAverage(W(:, j), X);
% update covariance
sigma_k = zeros(n, n);
Xm = bsxfun(@minus, X, mu(j, :));
for (i = 1 : m)
sigma_k = sigma_k + (W(i, j) .* (Xm(i, :)' * Xm(i, :)));
end
% Divide by the sum of weights.
sigma{j} = sigma_k ./ sum(W(:, j));
end
% Kiem tra hoi tu
if (mu == prevMu)
break
end
end
Anyone has answer?

Answers (0)

Asked:

on 1 Dec 2015

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!