An Intelligent Memory Architecture with LPDDR4 Extensions for Edge Generative AI Inference

Edge Generative AI (Gen-AI) inference has become prevalent in every aspect of the society. Present flagship edge products, however, fail to achieve high performance on Gen-AI models for the sake of limited bandwidth. An LPDDR4 compliant intelligent memory architecture is proposed in this paper to accelerate edge Gen-AI while maintaining regular LPDDR4 functions. Simulation shows the proposed architecture performs several times better than flagship products on Gen-AI, showing promise of this near-memory processing architecture.