Shape-Dependent Surface-Enhanced Raman Scattering under Modal Ultrastrong Coupling between Self-Assembled Gold Nanoparticles and Fabry–Pérot Cavities

Surface-enhanced Raman scattering (SERS) is a powerful tool for ultrasensitive molecular detection, yet its performance is critically dependent on the control of light-matter interactions. Here, we demonstrate that self-assembled gold nanoparticle films, composed of anisotropic gold nanotriangles (AuNTs) or isotropic gold nanospheres (AuNSs), exhibit shape-dependent SERS enhancement in a tailorable manner when coupled with a Fabry-Pérot (FP) cavity structure. The nanoparticle films were transferred onto semi-opened FP cavities consisting of a TiO₂ dielectric layer on a gold film. Systematic tuning of the TiO₂ thickness revealed modal ultrastrong coupling between the cavity and localized surface plasmon modes, yielding a peak splitting energy of ∼650 meV, as confirmed by reflection spectroscopy and finite-difference time-domain (FDTD) simulations. This modal ultrastrong coupling altered the extinction characteristics of the gold nanoparticle films, thereby significantly modulating the local electromagnetic fields and SERS signals. Shape-dependent SERS behaviors highlight the dominant influence of extinction-governed near-field intensity and hotspot distribution with additional contributions from surface chemistry factors associated with molecular accessibility. Under analyte-rich conditions, AuNT films exhibited strong SERS signals that were relatively insensitive to variations in the underlying TiO₂ thickness, owing to the broad hybrid band arising from multiple modal coupling. In contrast, AuNS films displayed a greater SERS enhancement but were more susceptible to shifts in the position of the sharp hybrid peaks. In analyte-limited conditions─where SERS is especially valuable due to the need for high sensitivity─AuNT films provided superior SERS signal quality, benefiting from their shape anisotropy, which promotes stronger hotspots and more efficient access to them. These insights contribute to the rational design of practical SERS platforms and highlight the potential of integrating self-assembled plasmonic films with FP cavities in advanced photonic and chemical sensing applications.