The covariance matrix in GMM go to singular. How can I fix it ? and Where is the problem ?
Show older comments
I am implementing GMM in matlab. But when I run that code, the covariance go to singular after 2 interations in EM-Algorithm. I don't know where is the problem. Data is 143-dimensional vector and about 300 observers. This is my code:
for (iter = 1:100)
fprintf(' EM Iteration %d\n', iter);
%%==========================a====================
%%STEP 3a: Expectation
% tinh lai trong so
pdf = zeros(m, k);
% For each cluster...
for (j = 1 : k)
% Tinh xac suat Gauss cho tat ca cac diem thuoc cluster 'j'.
pdf(:, j) = gaussianND(X, mu(j, :), sigma{j}); %<======this is my problem. Det(sigmal{j}) =0;
end
% pdf [m x k]
% phi [1 x k]
% pdf_w [m x k]
pdf_w = bsxfun(@times, pdf, phi);
% Divide the weighted probabilities by the sum of weighted probabilities for each cluster.
% sum(pdf_w, 2) -- sum over the clusters.
W = bsxfun(@rdivide, pdf_w, sum(pdf_w, 2));
%%===============================================
%%STEP 3b: Maximization
%%Calculate the probability for each data point for each distribution.
% store mean
prevMu = mu;
% Cap nhat cho moi cluster
for (j = 1 : k)
% update weight
phi(j) = mean(W(:, j), 1);
% update mean.
mu(j, :) = weightedAverage(W(:, j), X);
% update covariance
sigma_k = zeros(n, n);
Xm = bsxfun(@minus, X, mu(j, :));
for (i = 1 : m)
sigma_k = sigma_k + (W(i, j) .* (Xm(i, :)' * Xm(i, :)));
end
% Divide by the sum of weights.
sigma{j} = sigma_k ./ sum(W(:, j));
end
% Kiem tra hoi tu
if (mu == prevMu)
break
end
end
Anyone has answer?
Answers (0)
Categories
Find more on Big Data Processing in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!