Activity recognition in scientific experimentation using multimodal visual encoding
Vision-language foundation models augmented with domain-specific data can generate documentation of scientific experiments in real time based on egocentric videos of laboratory actions.
Authors
- Antonio Foncubierta
- Matteo Manica (ORCID: https://orcid.org/0000-0002-8872-0269)
- Andrea Giovannini (ORCID: https://orcid.org/0000-0002-2325-9299)
- Amol Thakkar (ORCID: https://orcid.org/0000-0003-0403-4067)
- I. Morales (ORCID: https://orcid.org/0000-0002-7467-3596)
- Gianmarco Gabrieli (ORCID: https://orcid.org/0000-0001-7279-7707)
- Dimitrios Christofidellis (ORCID: https://orcid.org/0000-0001-7757-8861)
- Federico Zipoli (ORCID: https://orcid.org/0000-0001-8345-9965)
- Mara Graziani
- Patrick Ruch (ORCID: https://orcid.org/0000-0003-0969-403X)
Institutions
- IBM Research - Zurich (CH)
Publication Details
- Journal
- Digital Discovery
- Published
- 2024-12-19
- DOI
- https://doi.org/10.1039/d4dd00287c
- Citations
- 1
- Primary Topic
- Robotics and Automated Systems
- Type
- article
- Field-Weighted Citation Impact
- 0.33