Towards Resource-Efficient Compound AI Systems
Compound AI Systems, integrating multiple interacting components like models, retrievers, and external tools, have emerged as essential for addressing complex AI tasks. However, current implementations suffer from inefficient resource utilization due to tight coupling between application logic and execution details, a disconnect between orchestration and resource management layers, and the perceived exclusiveness between efficiency and quality.
Authors
- Adam Belay (ORCID: https://orcid.org/0000-0003-2222-3611)
- Gohar Irfan Chaudhry (ORCID: https://orcid.org/0000-0002-1025-0009)
- Rodrigo Fonseca (ORCID: https://orcid.org/0000-0001-9662-2661)
- Esha Choukse (ORCID: https://orcid.org/0000-0003-0371-5522)
- Ricardo Bianchini (ORCID: https://orcid.org/0000-0001-5971-5084)
- Íñigo Goiri (ORCID: https://orcid.org/0000-0003-2591-4012)
Institutions
- Microsoft (United States) (US)
- Microsoft (Germany) (DE)
Publication Details
- Published
- 2025-05-14
- DOI
- https://doi.org/10.1145/3713082.3730377
- Citations
- 2
- Primary Topic
- Parallel Computing and Optimization Techniques
- Type
- article
- Field-Weighted Citation Impact
- 4.54
Funders
- Semiconductor Research Corporation
- Defense Advanced Research Projects Agency