OTAZ-DualPath: A Proprietary Inference Engine for Sparse Quantized Neural Networks on Low-Power CPUs

Running Large Language Models (LLMs) and deep neural networks on low-power central processing units (CPUs) remains a critical challenge for edge AI, embedded systems, and cost-sensitive AI infrastructure. Traditional quantization methods apply uniform precision reduction to all model weights, leading to accuracy degradation—particularly for outlier weights that carry disproportionate importance. Furthermore, existing solutions often require Graphics Processing Units (GPUs), calibration datasets, or suffer from high memory and compute overhead on CPU hardware. This paper presents OTAZ-DualPath, a proprietary inference execution engine designed specifically for low-power x86 CPUs (15W–65W TDP class). OTAZ-DualPath introduces a three-tier adaptive weight classification: Tier-1 preserves critical outlier weights in full FP32 precision, Tier-2 quantizes normal weights to Int8 using adaptive per-row scaling, and Tier-3 prunes unimportant weights to zero. All values are unified into a single execution stream processed by one optimized SIMD kernel (AVX2+FMA), eliminating branching overhead and dual-path execution. Illustrative theoretical results demonstrate that OTAZ-DualPath can achieve near-FP32 model accuracy with up to 7.7× memory compression on sparse models (90% sparsity), zero runtime memory allocations, and up to 5.5× parallelism efficiency improvement over standard OpenMP for small-batch inference. The engine requires no calibration data and performs post-training quantization in seconds. Note: All numerical examples are theoretical estimates based on specific experimental conditions (x86 CPU with AVX2+FMA, batch size = 1, specific model dimensions). Actual results may vary depending on hardware configuration, model architecture, and deployment scenario. OTAZ-DualPath is a proprietary commercial product available for licensing to AI companies, IoT manufacturers, embedded device OEMs, and edge computing providers. This preprint outlines the architecture, features, performance results, and comparison with existing quantization and inference systems.