Getting the centroids in KNN clasiification

Hi,

I am using a knn algorithm to perform supervised classification

My KNNClassification object is created as it follows :

fitcknn(obj.Table,obj.Response,"NSMethod","exhaustive","Distance","euclidean")

I wondered if it was possible to get the centroids coordinates as when I perform unlabelled clustring with kmeans.

Thanks in advance for your futur help.

Best regards

Mathieu

[Ayush]

Hi @Mathieu Chêne,

The k-Nearest Neighbors (KNN) algorithm is a supervised machine learning algorithm, which means it uses labeled data to make predictions or classify new examples. In contrast, K-means is an unsupervised clustering algorithm that groups similar data points into clusters based on some chosen distance metric.

In K-means, the centroids of the clusters can be obtained as a result of the clustering process. However, in KNN, there are no centroids as the algorithm simply compares the distance between new observations and pre-existing labeled observations.

Therefore, in short, it is not possible to get the centroids coordinates in KNN classification as there are no centroids in the algorithm.

Answer 1

[Divyam]

Hi @Mathieu Chêne,

Unlike the K-Means clustering algorithm, that explicitly computes centroids of clusters, the K-Nearest Neighbors (KNN) classification algorithm, does not calculate centroids as the algorithm is based on a distance-based method that classifies data points based on the closest training examples in the feature space.

However, if you want to calculate something akin to centroids for each class in your KNN model, you can compute the mean of the feature values for each class in your training data. This will provide a centroid-like representation for each class in your data derived from the labeled data used for training your KNN model.

Here's some code to help you with the same based on the assumption that "obj.Table" is your feature matrix and "obj.Response" is your class labels:

% Get unique class labels
classes = unique(obj.Response);

% Initialize the centroid matrix
centroids = zeros(length(classes), size(obj.Table, 2));

% Calculate the centroid for each class
for i = 1:length(classes)
    % Get indices for the current class
    classIdx = obj.Response == classes(i);
    
    % Compute the mean of the features for the current class
    centroids(i, :) = mean(obj.Table(classIdx, :), 1);
end