TensorSearch: Parallel Similarity Search on Tensors

Existing similarity search methods, often limited to scalar or vector data, struggle to identify complex patterns found in scientific datasets, such as 2D seismic events or 3D magnetic flux ropes. We introduce TensorSearch, a novel parallel similarity search paradigm designed to identify known patterns in high-dimensional tensors. By directly employing tensor representations, TensorSearch captures intricate pattern structures more effectively than traditional vector-based approaches. Furthermore, its parallel architecture optimizes cache and I/O operations, enabling efficient processing of large-scale scientific data. Our performance evaluations demonstrate that TensorSearch outperforms state-of-the-art vector-based systems like Milvus by up to 10x, and achieves up to a remarkable 55x advantage over custom solution developed in Matlab used by the domain scientists. In these tests, TensorSearch exhibits linear scalability, supporting up to 2240 CPU cores.

Authors

Institutions

Publication Details

Published
2024-12-15
DOI
https://doi.org/10.1109/bigdata62323.2024.10825798
Primary Topic
Computational Physics and Python Applications
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

TensorSearch: Parallel Similarity Search on Tensors

Avinash Nayak, Verónica Rodrı́guez Tribaldos, Kesheng Wu, Jonathan Ajo‐Franklin et al.
Computational Physics and Python Applications
article

TensorSearch: Parallel Similarity Search on Tensors

Avinash Nayak, Verónica Rodrı́guez Tribaldos, Kesheng Wu, Jonathan Ajo‐Franklin, Suren Byna, Alex Sim, Patrick Dobson, Qile Zhang, Fan Guo, Bin Dong
article en

Abstract

Existing similarity search methods, often limited to scalar or vector data, struggle to identify complex patterns found in scientific datasets, such as 2D seismic events or 3D magnetic flux ropes. We introduce TensorSearch, a novel parallel similarity search paradigm designed to identify known patterns in high-dimensional tensors. By directly employing tensor representations, TensorSearch captures intricate pattern structures more effectively than traditional vector-based approaches. Furthermore, its parallel architecture optimizes cache and I/O operations, enabling efficient processing of large-scale scientific data. Our performance evaluations demonstrate that TensorSearch outperforms state-of-the-art vector-based systems like Milvus by up to 10x, and achieves up to a remarkable 55x advantage over custom solution developed in Matlab used by the domain scientists. In these tests, TensorSearch exhibits linear scalability, supporting up to 2240 CPU cores.

Los Alamos National Laboratory (US), Lawrence Berkeley National Laboratory (US), GFZ Helmholtz Centre for Geosciences (DE), The Ohio State University (US), Rice University (US)
U.S. Department of Energy, Office of Energy, Office of Science, Technology Development, Geothermal Technologies Office
Openalex Percentile: Top 77%
Computational Physics and Python Applications
DOIOpen Access PDFOpenAlex
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

TensorSearch: Parallel Similarity Search on Tensors — Avinash Nayak, Verónica Rodrı́guez Tribaldos, et al. · (2024) | TGRS Research Map | TGRS