Detecting Outliers in Context of Clustering Imbalanced Categorical Data

Unsupervised models are becoming increasingly common in business processes. They are extremely effective in cases where we don't have a clearly defined decision class or the data contains anomalies that are hard to identify. The problem emerges in the effective processing of categorical data. Recently, many new approaches have been designed to analyze this data type. Still, most of them do not address the issue of unbalanced datasets, which is extremely difficult to catch when dealing with unlabeled data. Moreover, it is sometimes challenging to determine when abnormal observations represent a small data cluster and when they are already anomalies. This research analyzes several less popular algorithms that solve this problem and automatically place abnormal observations into separated clusters. We have shown that such methods are much better at clustering unbalanced data but also perfectly detect outliers in categorical datasets.

Authors

Institutions

Publication Details

Journal
Proceedings of the International Conference on Information Systems Development
Published
2024-09-09
DOI
https://doi.org/10.62036/isd.2024.35
Primary Topic
Advanced Statistical Methods and Models
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Detecting Outliers in Context of Clustering Imbalanced Categorical Data

Weronika Łazarz, Agnieszka Nowak-Brzezińska
Proceedings of the International Conference on Information Systems Development
Advanced Statistical Methods and Models
article

Detecting Outliers in Context of Clustering Imbalanced Categorical Data

Weronika Łazarz, Agnieszka Nowak-Brzezińska
article en

Abstract

Unsupervised models are becoming increasingly common in business processes. They are extremely effective in cases where we don't have a clearly defined decision class or the data contains anomalies that are hard to identify. The problem emerges in the effective processing of categorical data. Recently, many new approaches have been designed to analyze this data type. Still, most of them do not address the issue of unbalanced datasets, which is extremely difficult to catch when dealing with unlabeled data. Moreover, it is sometimes challenging to determine when abnormal observations represent a small data cluster and when they are already anomalies. This research analyzes several less popular algorithms that solve this problem and automatically place abnormal observations into separated clusters. We have shown that such methods are much better at clustering unbalanced data but also perfectly detect outliers in categorical datasets.

Proceedings of the International Conference on Information Systems Development
University of Silesia in Katowice (PL)
Openalex Percentile: Top 84%
Advanced Statistical Methods and Models
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.