Open Set Recognition using Dissimilarity Vectors

A persistent challenge in machine learning is the identification of samples from unknown classes within test datasets—a task essential for improving class recognition and open set classification. This study investigates dissimilarity-based techniques as a means to address this issue, leveraging their conceptual alignment with open space scenarios, where samples may not belong to any known class. We propose a handcrafted approach to accurately detect such samples. Experiments were conducted on a synthetic feature dataset arranged in a grid, incorporating varying inter-class distances and noise levels. The method evaluates whether a sample exhibits sufficient similarity to known classes; lacking this, it is flagged as potentially unknown. For comparison, we also assess a probability thresholding strategy using conventional classifiers. The dissimilarity-based approach achieved a misclassification rate of approximately 16%, compared to nearly 37% using probability thresholds. Results indicate that dissimilarity methods are more robust in open set contexts, maintaining consistent performance despite variations in noise and class separation—an advantage not observed with probability-based method.