Problems clustering with kmeans
13 views (last 30 days)
Show older comments
I have a data set that is x,y,v. Each vector is quite long, ~2 million rows. Also, the data is scattered, i.e. not at regular x,y intervals/spacings. An x-y plot of a small section is included here. When plotting x-y you can see groupings of 9 data points each. There will be ~200K of these groupings. I need to cluster the data set and get an average v value for each cluster of 9 points.
I have used kmeans clustering:
[idx,CC] = kmeans([x,y],round(length(x)/9),'Replicates',10,'Options',statset('UseParallel',1));
This works for maybe 66% of the clusters. In other words, about 66% of the clusters it finds are composed of 9 data points as desired. The other clusters are somewhat smaller or larger in the amount of data points comprising them. Also attached is a histogram of the amounts of data points in the clusters that kmeans returns. The large peak at 9 is what I want and is 66% of the total values...
If I know how many points should be in each cluster (9) and I know the intra-point distance within the cluster (because the spacing of the 9 data points within a cluster is constant), is there a way to improve upon these results of kmeans? Can I stipulate that a cluster must have 9 points within it? Can I stipulate that a cluster can have a distance between its members that is no more than a prescribed value?
Other things I'm considering is looping over a kmeans calculation and each time just keeping the clusters that have the 9 points--repeating the kmeans for the data set with those entries removed. This may work but it seems there should be a better way considering I know a decent amount about the data structure.
Thanks for any suggestions!
5 Comments
the cyclist
on 3 Sep 2022
Edited: the cyclist
on 3 Sep 2022
Glad it worked out for you.
I'm curious what inputs worked for you, using dbscan (and if success seemed dependent on getting them right).
Accepted Answer
More Answers (1)
See Also
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!