BEYOND EXTERNAL CONTROL: HYPERNETWORK-DRIVEN PARAMETER EDITING FOR MULTI-MODAL IMAGE GENERATION

Current controllable image generation methods predominantly rely on external architectural modifications, such as auxiliary control networks, which require substantial computational overhead and struggle to unify diverse control modalities including text, pose, depth, and sketches. These approaches fundamentally limit scalability and real-time applicability due to their additive nature and complex multi-condition integration challenges. We introduce HyperEdit, a novel hypernetwork-driven framework that achieves multi-modal controllable generation through dynamic parameter perturbation of pre-trained diffusion models, moving beyond external control paradigms toward intrinsic model adaptation. Our approach employs a unified hypernetwork that learns to map diverse control conditions—ranging from textual descriptions and pose skeletons to depth maps and edge sketches—into targeted parameter perturbations, enabling seamless integration of multiple modalities without architectural modifications to the base model. Through systematic perturbation discovery on carefully constructed condition-image pairs and progressive parameter injection strategies, HyperEdit demonstrates remarkable efficiency gains, achieving up to 6× faster inference compared to existing methods while requiring significantly fewer parameters. Extensive experiments across diverse control scenarios show that our unified framework not only maintains generation quality comparable to specialized control methods but also enables novel capabilities such as real-time condition mixing, dynamic editing strength adjustment, and reversible modifications. This work establishes a new paradigm for controllable generation that bridges the gap between research innovation and practical deployment requirements.