Scaling Parallel Algorithms to Massive Datasets using Multi-SSD Machines

It is now possible in principle to build relatively inexpensive multi-core servers equipped with dozens of terabytes, to even petabytes of local NVMe SSD storage. This raises a natural question of what can be done with such machines, and how many parallel storage devices are required to quickly compute over data that is much larger than main memory? Can we design algorithms for such machines that achieve nearly in-memory performance while gracefully scaling to datasets that are much larger than main memory?

Authors

Institutions

Publication Details

Published
2025-07-16
DOI
https://doi.org/10.1145/3694906.3743308
Primary Topic
Advanced Data Storage Technologies
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Scaling Parallel Algorithms to Massive Datasets using Multi-SSD Machines

H.N. Li, Laxman Dhulipala, Jamshed Khan
Advanced Data Storage Technologies
article

Scaling Parallel Algorithms to Massive Datasets using Multi-SSD Machines

H.N. Li, Laxman Dhulipala, Jamshed Khan
article en

Abstract

It is now possible in principle to build relatively inexpensive multi-core servers equipped with dozens of terabytes, to even petabytes of local NVMe SSD storage. This raises a natural question of what can be done with such machines, and how many parallel storage devices are required to quickly compute over data that is much larger than main memory? Can we design algorithms for such machines that achieve nearly in-memory performance while gracefully scaling to datasets that are much larger than main memory?

University of Maryland, College Park (US)
National Science Foundation
Openalex Percentile: Top 81%
Advanced Data Storage Technologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.