Bitwise-Identical Machine Learning Across GPUs and CPUs

The same machine-learning workload can produce different bits on different processors, changing predictions, learned models and subsequent training updates. Mojolearn is a machine-learning library whose default configuration produces bitwise-identical results across verified Apple, NVIDIA and AMD GPUs, and Arm and x86-64 CPUs. Results agree bit for bit, not merely within a numerical tolerance. It supports training and inference for over 100 machine-learning algorithms in 13 families, from decision trees to neural networks. Users can also run and train custom models built from supported operations under the same cross-vendor identity guarantee. Neural models trained in other frameworks can be served with identical outputs across vendors, and training workloads can be handed off between GPU vendors mid-run, or shared by GPUs of different vendors at once, while maintaining bitwise identity. Using Mojolearn, I trained two instances of a 162-million-parameter GPT-3 Small-shaped model along different hardware routes collectively spanning Apple, NVIDIA and AMD GPUs, and maintained bitwise identity at all 5,000 optimizer steps. Neural training and inference run in full FP32 floating-point arithmetic, without reducing the computation to integers. To my knowledge, I am the first to enable these capabilities. Evidence-0.8.55.zip contains the release 0.8.55 evidence files exactly as submitted for review; in them, the library is named "anonlib", which is Mojolearn. MANIFEST.sha256 lists the SHA-256 of every file.