Let the Augmentation Path Speak

In this work, we introduce a novel tool AugPath, for discovering relevant tabular data from a data lake to augment a base table, thereby supporting a wide range of downstream tasks, such and training data augmentation and table question answering (Q&A). Unlike traditional methods, such as unionable and joinable table search, which rely on one-hop and direct relationships for table augmentation, AugPath focuses on augmentation paths-sequences of interconnected tables linked to the base table through both direct and indirect relationships. By exploring these multi-step connections, AugPath enables more comprehensive and informative data augmentations, uncovering insights that traditional methods might overlook. Specifically, given a base table and a natural language statement, AugPath aims to identify and rank augmentation paths based on three core components: (1) a User Intent Extractor that utilizes a large language model (LLM) to analyze user intent and convert natural language statements into specific and actionable data requirements; (2) a Candidate Path Retriever that identifies all possible augmentation paths within a data lake that can be used for augmenting the base table; and (3) a Path Utility Evaluator that scores and ranks the paths using a novel utility metric, which considers both their relevance to the query and the diversity of the data they provide. AugPath enables users to discover augmentation paths that enhance common downstream tasks, such as training data augmentation and single/multiple table Q&A.