Moving Fast and Breaking Clients? An Empirical Study of API Breaking Changes in Python LLM SDKs and Agent Frameworks

Abstract Context. LLM-based applications depend on a young layer of libraries: provider SDKs such as OpenAI’s and Anthropic’s, and frameworks such as LangChain and CrewAI. Practitioners say these libraries change constantly, but this has not been measured. Objective. We measure how often LLM libraries break their public APIs, whether version numbers and deprecations warn users, and how dependent packages respond, compared with mature Python libraries. Method. We statically compared consecutive releases of 24 LLM libraries and 16 mature libraries (5,468 releases, January 2023 to October 2026), validated 160 detected breaking changes, and analyzed 8,380 dependency declarations from the 15,000 most downloaded PyPI packages. Results. LLM libraries released 2.3 times as often (median 31.4 vs. 13.7 releases per year) and broke their API in more releases (35.5% vs. 21.7%), four times as often relative to API size. Most breaking changes arrived in releases that semantic versioning does not permit to break, and only 2.9% of objects removed by LLM libraries were previously marked deprecated (8.5% for mature libraries). About 70% of detected breaking changes were real in both groups. Dependents bounded or pinned 50.8% of dependencies on LLM libraries (30.7% for mature ones), and 23.0% excluded the latest release (5.7%). Conclusions. The instability is real but concentrated in frameworks rather than provider SDKs, and in versioning and deprecation practices as much as in the rate of change.