Can foundation language models predict fluid dynamics?

The application of deep learning-based, data-driven methods to fluid dynamics problems has attracted significant interest in recent years, due to their potential for faster flow predictions (model inferences) and reduced computational requirements compared to traditional Computational Fluid Dynamics (CFD) methods. Despite their success, existing supervised deep learning approaches typically require extensive model design and large amounts of high-quality training data to achieve satisfactory performance for each specific fluid flow problem. The emergence of foundation models—pre-trained on large-scale, multidisciplinary datasets and capable of solving a wide range of downstream tasks—raises the question of whether their capabilities can extend to scientific domains such as fluid dynamics. Most foundation models to date have been rooted in language tasks. This study investigates a specific instantiation of a foundation model, the Large Language Model Meta AI (Llama) 3 model, for predicting fluid flows with varying dynamical complexities. Results show that the Llama 3 foundation model, although originally designed for natural language processing, can be applied to fluid dynamics problems with simple engineering adaptations and without fine-tuning its pre-trained weights. This approach achieves improved accuracy and robustness compared to conventional fully connected or recurrent neural network models with comparable capacities and equivalent training data. Furthermore, fine-tuning the model by injecting problem-specific knowledge into the pre-trained weights further enhances its performance. These findings suggest that foundation models hold promise for fast inference and deployment of solutions to fluid dynamics problems. Whether unified with models from other domains (e.g., language models) or developed specifically for fluid dynamics, such models could become powerful tools for improving predictive accuracy and computational efficiency in engineering and scientific applications. The code is accessible at https://gitlab.com/houWGW/llm_for_fluid_dynamics . • Adapted Llama 3 to fluid dynamics benchmarks, outperforming traditional CAE-MLP and CAE-LSTM models. • Demonstrated that pre-trained LLM weights contain generalisable knowledge applicable to dynamical systems. • Validated Llama 3’s effectiveness across a range of subsonic and supersonic benchmarks.