When Vowels Don't Help: Empirical Limits of Vowel-Consonant Separation for Compression and LLM Pipelines
We study a simple linguistic transform that separates English vowels from consonants (MVD, Maharashtran Vowel Disappearance) and ask whether it helps three common goals: reversible representation, lossless compression with standard codecs, and cheaper LLM/RAG pipelines via lossy vowel dropping. The reversible split works. Lossless MVD serialization plus gzip/bz2/zlib is worse than compressing raw text (about -120% relative improvement on short synthetic corpora; still strongly negative on real books). Lossy vowel dropping before gzip does help: about +14% mean savings on synthetic text and +20.5% mean savings on real public-domain prose at >=200 KB. The same lossy transform fails for RAG-relevant proxies: tiktoken (cl100k_base) token counts increase by an unweighted mean of 76.8%, and sentence-transformer cosine similarity collapses to 0.17 (96.7% of pairs below 0.6). We conclude that vowel dropping is a narrow byte-level compression trick, not a path to LLM token or embedding efficiency. Byte savings do not transfer to BPE tokenizers or pretrained embedders that rely on recognizable subword units. Code: Github_MVD
Authors
- Syeda Fayeqa Nabeel
- Mohammed Abdul Lateef
- Mohammed Afeefuddin
- Mohammed Abdul Moid
- Nuzhath Bagum
- Amar Bin Abdullah Bawazeer
- Syed Liyaqat Ahmed Ali
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-14
- DOI
- https://doi.org/10.5281/zenodo.23376959
- Primary Topic
- Algorithms and Data Compression
- Type
- preprint