ConfSum: Towards Automatic Summarization of Network-scale Operational Intents from Device Configurations

When network operators need to understand the high-level intent behind a network's existing device configurations, they must engage in a tedious and error-prone process of manually reverse-engineering the low-level commands. We propose Configuration Intent Summarization (CIS), a new task that aims to automate this process by generating human-readable summaries of the intents embedded across a network's configurations. CIS is challenging due to the diversity of intents, the semantic gap between device-specific configurations and network-wide intents, and the need to reason about interactions between multiple devices' configurations. We present ConfSum, a system that addresses these challenges by leveraging the unique ability of large language models (LLMs) to parse semi-structured configuration files and summarize them in natural language. However, the full CIS task requires reasoning about device interactions and other complexities that are beyond the capabilities of LLMs alone. To enhance the LLM's robustness to these challenges, ConfSum introduces novel techniques for retrieving relevant examples to augment LLM prompts, decomposing the generation process to handle multi-device intents, and integrating with formal validation tools. Our experiments demonstrate that Conf-Sum achieves high intent coverage while generating summaries that match the quality of human experts.