Can I use a square matrix for the GMM cluster?

Hello,

I would like to use a square matrix for cluster analysis using the Gaussian Mixture Model.

I would like to use the soft variant for this. Unfortunately, I get the following error messages:

Error using gmdistribution.fit

X must have more rows than columns.

Error in fitgmdist (line 135)

gm = gmdistribution.fit(X,k,varargin{:});

Hence the question: How can I change the code to cluster the 50x50 matrix?

%Code

>> % Erzeugen einer 50x50 Matrix mit zufälligen Werten

X = rand(50, 50);

% GMM-Clusteranalyse durchführen

maxNumClusters = 10; % Maximale Anzahl von Clustern

BIC = zeros(1, maxNumClusters);

for k = 1:maxNumClusters

gm = fitgmdist(X, k);

BIC(k) = gm.BIC;

end

% Bestimmung der optimalen Anzahl von Clustern basierend auf dem BIC

[~, optimalNumClusters] = min(BIC);

% GMM-Clusteranalyse mit optimaler Anzahl von Clustern durchführen

gm = fitgmdist(X, optimalNumClusters);

% Cluster-Zuweisungen erhalten

idx = cluster(gm, X);

% Visualisierung der Cluster

figure;

scatter(X(:,1), X(:,2), 10, idx, 'filled');

xlabel('Feature 1');

ylabel('Feature 2');

title(sprintf('GMM Cluster mit %d Clustern', optimalNumClusters));

colorbar;

Error using gmdistribution.fit

X must have more rows than columns.

Error in fitgmdist (line 135)

gm = gmdistribution.fit(X,k,varargin{:});

Answer 1

[Sudarsanan A K]

Hi Alexander,

To perform cluster analysis on a x square matrix using the Gaussian Mixture Model (GMM) without encountering the X must have more rows than columns error, you can use Principal Component Analysis (PCA) to reduce the dimensionality of your data. This approach ensures that the number of features (columns) is less than the number of observations (rows), making the data suitable for GMM clustering. Here's how you can modify your code to include PCA before applying GMM:

% Generating a 50x50 matrix with random values
X = rand(50, 50);

% Reducing dimensions with PCA
[coeff, score, ~, ~, explained] = pca(X);
% Determining the number of components that explain 95% of the variance
numComponents = find(cumsum(explained) >= 95, 1, 'first');
X_reduced = score(:, 1:numComponents);

% Performing GMM cluster analysis
maxNumClusters = 10; % Maximum number of clusters
BIC = zeros(1, maxNumClusters);
options = statset('MaxIter', 1000); % Increasing the maximum number of iterations
regularizationValue = 1e-5; % Adding a regularization value to improve numerical stability

for k = 1:maxNumClusters
    try
        gm = fitgmdist(X_reduced, k, 'Options', options, 'RegularizationValue', regularizationValue);
        BIC(k) = gm.BIC;
    catch ME
        warning('Failed to converge for k=%d: %s', k, ME.message);
        BIC(k) = inf; % Set BIC to infinity to discard this cluster value
    end
end

% Determining the optimal number of clusters based on the BIC
[~, optimalNumClusters] = min(BIC);

% Checking if a valid number of clusters was found
if isfinite(optimalNumClusters)
    % Performing GMM cluster analysis with the optimal number of clusters
    gm = fitgmdist(X_reduced, optimalNumClusters, 'Options', options, 'RegularizationValue', regularizationValue);

    % Obtaining cluster assignments
    idx = cluster(gm, X_reduced);

    % Visualizing the clusters (adjusted for reduced dimensions)
    figure;
    scatter(X_reduced(:,1), X_reduced(:,2), 10, idx, 'filled');
    xlabel('Feature 1');
    ylabel('Feature 2');
    title(sprintf('GMM Clusters with %d Clusters', optimalNumClusters));
    colorbar;
else
    error('GMM failed to converge for any cluster value.');
end

Applying PCA reduces your dataset's dimensionality, concentrating on significant variance, making it suitable for GMM clustering. The code incorporates regularization to enhance the numerical stability of covariance matrices, ensuring successful cluster analysis.

For more information about clustering Gaussian mixture data, see these examples:

• https://mathworks.com/help/stats/clustering-using-gaussian-mixture-models.html
• https://mathworks.com/help/stats/cluster-gaussian-mixture-data-using-soft-clustering.html

For more information on PCA, refer to the following resource:

• https://mathworks.com/help/stats/principal-component-analysis-pca.html

I hope this helps!