AI Insight
This paper presents a modified k-means clustering algorithm that uses two distance measures simultaneously: within-cluster distance (WCD) and inter-cluster distance (ICD), rather than the traditional single distance metric. The optimal number of clusters is determined using the Calinski-Harabasz criterion. Testing on synthetic data and UCI repository benchmark datasets demonstrates improved clustering accuracy and better handling of outliers compared to traditional k-means methods.
Why it matters
Improved clustering algorithms have broad applications across data science, from customer segmentation in business to pattern recognition in scientific research. The enhanced ability to correctly classify outliers could be particularly valuable in fields like anomaly detection, medical diagnostics, and quality control where edge cases are critically important.
Understand the Science
Abstract: Clustering algorithms have long been the topic of research, representing the more popular side of unsupervised learning. Since clustering analysis is one of the best ways to find some clarity and structure within raw data, this paper explores a novel approach to k-means clustering. Here we present a k-means clustering algorithm that takes both the within cluster distance (WCD) and the inter cluster distance (ICD) as the distance metric to cluster the data into k clusters pre-determined by the Calinski-Harabasz criterion in order to provide a more robust output for the clustering analysis. The idea with this approach is that by including both the measurement metrics, the convergence of the data into their clusters becomes solidified and more robust. We run the algorithm with some synthetically produced data and also some benchmark data sets obtained from the UCI repository. The results show that the convergence of the data into their respective clusters is more accurate by using both WCD and ICD measurement metrics. The algorithm is also better at clustering the outliers into their true clusters as opposed to the traditional k means method. We also address some interesting possible research topics that reveal themselves as we answer the questions we initially set out to address.
Source: A novel k-means clustering approach using two distance measures for Gaussian data