Outliers detection in datasets with misclassified objects
Background: The problem of outliers detection is one of the important problems in Data Mining. Here, outliers are considered as initially misclassified objects of the dataset. Such objects in small datasets can seriously interrupt the process of classification. This paper describes an algorithm of censoring such data, focusing only on the local characteristics of objects in the dataset. Methods: Censoring procedure in a fixed feature space consists of sequential removals of objects, which deteriorate the quality of dataset description (a value of classes’ separability) in the strongest way. This value depends on the number of objects in the dataset and similarity of objects with their class in competition with the rival class. To evaluate the similarity of the object with class in competition with class 𝐵, the ternary relative measure called the function of rival similarity (FRiS-function) is used. Results: The proposed algorithm was tested on a wide range of model problems. Accuracy of nearest neighbors classification before and after outliers elimination from the datasets was in use to estimate efficiency of the censoring algorithm. In the most tasks, it is appeared to be improvement in classification accuracy after censoring. Analysis of objects which were recognized as outliers showed up to 96% sensitivity and 99% specificity. Concluding Remarks: According to the obtained results, it is possible to conclude that the objects, which were deleted before the inflection point of the classes separability function, usually distort the structure of the data. Therefore, their exclusion from the analyzed dataset increases the reliability of recognition.