Orthogonal Adaptative Networks for Structured Word Embedding Compression

This comparative study represents the Orthogonal Adaptive Neural Network (OA-NN), a novel dimensionality reduction method that synergizes orthogonal linear projections with adaptive nonlinear transformations to preserve both local and global structures in high-dimensional data. The dataset which we have used in this paper, Glove (Global Vectors for Word Representation), is an embedded dataset with 400000 embedded word to vectors. We curated a subset of 1000 word to vectors with the mentioned dataset from 400000 (word to vectors) to reduce the time duration and balance between comprehensiveness and computational efficiency. The proposed method is rigorously compared against three established techniques: principal Component Analysis (PCA), t-Distributed Stochastic Neighbor Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP). we have used quantitative metrics (MSE, Trustworthiness, Continuity) and qualitative visual analysis for testing OA-NN method and and also for comparison,the OA-NN showed a (2-5) better continuity rate, and an equal continuity rate compared to UMAP while applying Glove 300D dataset.