FastFlow-Python: Parallel Building Blocks in Python Through FastFlow Integration
We present FastFlow-Python, a framework that brings parallelism to Python for stream-processing applications. FastFlow-Python enables developers to build high-throughput, low-latency data-flow networks by instantiating high-level, ready-to-use parallel building blocks. Built on the C++ FastFlow library, it leverages Python bindings via the C/Python API to efficiently manage parallel execution using both subinterpreters and multiprocessing, all abstracted by the framework. We demonstrate the performance benefits of FastFlow-Python through a comparative analysis with a pure Python stream-processing implementation, highlighting its effectiveness in overcoming the limitations imposed by the Global Interpreter Lock (GIL). Experimental results show almost linear scalability when increasing the number of workers.
Authors
- Jacopo Massa (ORCID: https://orcid.org/0000-0002-5255-537X)
- Massimo Torquati (ORCID: https://orcid.org/0000-0001-6323-3459)
- Patrizio Dazzi (ORCID: https://orcid.org/0000-0001-8504-1503)
- Matteo Della Bartola
Institutions
- University of Pisa (IT)
Publication Details
- Published
- 2025-09-23
- DOI
- https://doi.org/10.1109/ic2e65552.2025.00046
- Primary Topic
- Computational Physics and Python Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00