SkyServe: Serving AI Models across Regions and Clouds with Spot Instances
Recent years have witnessed an explosive growth of AI models. The high cost of hosting AI services on GPUs and their demanding service requirements, make it timely and challenging to lower service costs and guarantee service quality. While spot instances have long been offered with a large discount, spot preemptions have discouraged users from using them to host model replicas when serving AI models.
Authors
- Wei-Lin Chiang (ORCID: https://orcid.org/0009-0009-0105-723X)
- Romil Bhardwaj (ORCID: https://orcid.org/0000-0002-7314-1643)
- Ziming Mao (ORCID: https://orcid.org/0000-0001-7738-2498)
- Scott Shenker (ORCID: https://orcid.org/0000-0002-1357-7533)
- Ion Stoica (ORCID: https://orcid.org/0000-0002-5373-0088)
- Zhanghao Wu (ORCID: https://orcid.org/0000-0003-2352-4002)
- Tyler Griggs
- Zongheng Yang (ORCID: https://orcid.org/0000-0001-8716-8743)
- Tian Xia
Institutions
- International Computer Science Institute (US)
- Berkeley College (US)
- University of California, Berkeley (US)
Publication Details
- Published
- 2025-03-26
- DOI
- https://doi.org/10.1145/3689031.3717459
- Citations
- 5
- Primary Topic
- Scientific Computing and Data Management
- Type
- article
- Field-Weighted Citation Impact
- 20.84