The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understanding

Egocentric video has seen increased interest in recent years, as it is used in a range of areas. However, most existing datasets are limited to a single perspective. In this paper, we present the CASTLE 2024 dataset, a multimodal collection containing ego- and exo-centric (i.e., first- and third-person perspective) video and audio from 15 time-aligned sources, as well as other sensor streams and auxiliary data. The dataset was recorded by volunteer participants over four days in a common location and includes the point of view of 10 participants, with an additional 5 fixed cameras providing an exocentric perspective. The entire dataset contains over 600 hours of UHD video recorded at 50 frames per second. In contrast to other datasets, CASTLE 2024 does not contain any partial censoring, such as blurred faces or distorted audio. The dataset is available via https://castle-dataset.github.io/.

Authors

Institutions

Publication Details

Published
2025-10-25
DOI
https://doi.org/10.1145/3746027.3758199
Primary Topic
Multimodal Machine Learning Applications
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understanding

Cathal Gurrin, Klaus Schoeffmann, Onanong Kongmeesub, Ly-Duyen Tran et al.
Multimodal Machine Learning Applications
article

The CASTLE 2024 Dataset: Advancing the Art of Multimodal Understanding

Cathal Gurrin, Klaus Schoeffmann, Onanong Kongmeesub, Ly-Duyen Tran, Hoang-Bao Le, Luca Rossetto, Graham Healy, Minh–Triet Tran, Quang-Linh Tran, Duc‐Tien Dang‐Nguyen, Werner Bailer, Stevan Rudinac, Florian Spieß, Björn Þór Jónsson
article en

Abstract

Egocentric video has seen increased interest in recent years, as it is used in a range of areas. However, most existing datasets are limited to a single perspective. In this paper, we present the CASTLE 2024 dataset, a multimodal collection containing ego- and exo-centric (i.e., first- and third-person perspective) video and audio from 15 time-aligned sources, as well as other sensor streams and auxiliary data. The dataset was recorded by volunteer participants over four days in a common location and includes the point of view of 10 participants, with an additional 5 fixed cameras providing an exocentric perspective. The entire dataset contains over 600 hours of UHD video recorded at 50 frames per second. In contrast to other datasets, CASTLE 2024 does not contain any partial censoring, such as blurred faces or distorted audio. The dataset is available via https://castle-dataset.github.io/.

Reykjavík University (IS), Joanneum Research (AT), University of Basel (CH), Ho Chi Minh City University of Science (VN), Amsterdam University of the Arts (NL), University of Klagenfurt (AT), Dublin City University (IE), University of Bergen (NO), University of Amsterdam (NL)
National Science Foundation, Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
Openalex Percentile: Top 70%
Multimodal Machine Learning Applications
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.