Finding What You're Looking For: A Distribution-Aware Dataset Search Engine in Action

The growing volume of academic, commercial, and governmental datasets distributed across countless independent repositories calls for dataset search engines that can answer queries only using publicly shared metadata instead of relying on raw data access. However, the keyword search interfaces of existing metadata-based search engines fail to capture complex user needs, such as distributional requirements, thereby limiting their effectiveness. In this demonstration, we present the first end-to-end system for distribution-aware dataset search over decentralized data repositories. Our prototype combines existing search techniques with recently proposed percentile predicates to provide more powerful query capabilities. Based on our novel Dataset Query Language and a distribution-aware index, the system enables efficient, flexible search without access to raw data. To demonstrate its utility, we curated over 150,000 profiles of tabular datasets from Kaggle and enriched them with statistical information, enabling attendees to explore distribution-aware search and the trade-offs involved in system configuration.

Authors

Institutions

Publication Details

Published
2025-06-17
DOI
https://doi.org/10.1145/3722212.3725104
Citations
1
Primary Topic
Web Data Mining and Analysis
Type
article
Field-Weighted Citation Impact
3.25
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Finding What You're Looking For: A Distribution-Aware Dataset Search Engine in Action

Volker Markl, Kaustubh Beedkar, Lennart Behme, Leonard Geißler et al.
1 citations
Web Data Mining and Analysis
3.25
article

Finding What You're Looking For: A Distribution-Aware Dataset Search Engine in Action

Volker Markl, Kaustubh Beedkar, Lennart Behme, Leonard Geißler, Emil Badura, Benjamin Ueber, Pratham Agrawal
article en
1 citations

Abstract

The growing volume of academic, commercial, and governmental datasets distributed across countless independent repositories calls for dataset search engines that can answer queries only using publicly shared metadata instead of relying on raw data access. However, the keyword search interfaces of existing metadata-based search engines fail to capture complex user needs, such as distributional requirements, thereby limiting their effectiveness. In this demonstration, we present the first end-to-end system for distribution-aware dataset search over decentralized data repositories. Our prototype combines existing search techniques with recently proposed percentile predicates to provide more powerful query capabilities. Based on our novel Dataset Query Language and a distribution-aware index, the system enables efficient, flexible search without access to raw data. To demonstrate its utility, we curated over 150,000 profiles of tabular datasets from Kaggle and enriched them with statistical information, enabling attendees to explore distribution-aware search and the trade-offs involved in system configuration.

Technische Universität Berlin (DE), Indian Institute of Technology Delhi (IN)
Openalex Percentile: Top 8%
Web Data Mining and Analysis
3.25
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Finding What You're Looking For: A Distribution-Aware Dataset Search Engine in Action — Volker Markl, Kaustubh Beedkar, et al. · (2025) | TGRS Research Map | TGRS