Buoyancy–Marangoni convection in open trapezoidal cavities filled with NEPCM nanofluids

Purpose This paper aims to investigate coupled buoyancy–Marangoni convection in open trapezoidal cavities filled with nano-encapsulated phase change material (NEPCM) suspensions. Design/methodology/approach A comprehensive numerical investigation is conducted using the Galerkin finite element method with a penalty formulation to eliminate pressure. The dimensionless governing equations are solved for three distinct geometries: trapezoidal with negatively sloped hot wall (trap-), square and trapezoidal with positively sloped hot wall (trap+). The NEPCM suspension is modeled via an apparent heat capacity method capturing latent effects within a prescribed fusion temperature range. Parametric studies span Marangoni numbers (−5000≤Ma≤8000), NEPCM concentrations (0.0≤ϕ≤0.05), Prandtl numbers (0.054≤Pr≤6.2) and fusion temperatures (0.05≤ΘF≤0.3). Findings The trap + geometry uniquely optimizes coupling between buoyant and thermocapillary forces, sustaining the strongest circulation at high Marangoni numbers and promoting the most extensive phase change region. NEPCM concentration significantly boosts heat transfer in square and trap + cavities at moderate Rayleigh numbers, yet its effect diminishes in trap– geometries and under strong Marangoni dominance. Research limitations/implications The present findings are applicable to steady-state, laminar flow regimes, providing a foundational understanding for low-to-moderate Reynolds number applications such as small-scale solar receivers and open-channel micro-coolers. However, several limitations must be acknowledged: the numerical model assumes a two-dimensional domain, which neglects potential three-dimensional end-wall effects and vortex stretching that may occur in wider industrial cavities. Practical implications The findings provide design guidance for open-cavity thermal systems such as solar receivers. Originality/value By analyzing the interaction between surface-tension-driven flow and buoyancy forces, this work provides a detailed physical interpretation of how interfacial transport mechanisms govern thermal storage density. Unlike conventional enclosed configurations, the combination of an open boundary, trapezoidal geometry and NEPCM suspensions introduces a distinct hydrodynamic environment in which Marangoni-induced shear at the free surface significantly alters particle dynamics. This study presents the first comprehensive analysis of coupled buoyancy–Marangoni convection in such systems, while also identifying optimal operating conditions for enhanced heat transfer.