SGK Empirical Validation: Search for Optimal Application Domains
The SGK-Attention mechanism with ReLU causes network degeneracy in this specific architecture (ViT on MNIST). However, this does not invalidate the theoretical framework (SGK v1.2) — it indicates that this particular implementation is not optimal for this task.
Authors
- Adrian Pietrzela (ORCID: https://orcid.org/0009-0005-9044-9331)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-06-30
- DOI
- https://doi.org/10.5281/zenodo.21078743
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00