Lost in Chemical Space: Local Open-Source Large Language Models Display Model-Dependent Adherence to Reference Data in Molecular Generation

Large language models (LLMs) can generate drug-like molecules and may be used locally in drug discovery projects where confidentiality and intellectual-property constraints preclude the use of external APIs. Weight quantization enables deployment of large open-source LLMs on limited hardware, but its impact on molecular generation and on the ability of models to follow reference chemical spaces remains poorly understood. Here, we evaluated Gemma-4-31B, GPT-OSS-120B, Mistral-Small-4-119B, and Qwen-3.6-35B-A3B at the BF16 (original precision), Q8, Q6, and Q4 quantization levels in two molecular generation tasks. The models were prompted either to generate small molecules without constraints or to generate drug-like molecules conditioned on 69 reference kinase inhibitors provided in-context. Quantization increased inference throughput in a model-dependent manner, with the largest gains, up to 24-fold increase in performance, occurring for Gemma-4-31B and Qwen-3.6-35B-A3B. In the task of unconstrained generation, model quantization did not affect molecular validity, uniqueness, drug-likeness, diversity and synthesizability. In the task of reference-guided generation, however, the models displayed markedly different levels of adherence to the provided chemical space. Gemma-4-31B was the only model that consistently generated small molecules similar to the reference compounds at all precision levels. Mistral-Small-4-119B and Qwen-3.6-35B-A3B failed to generate small molecules according to the reference chemical space. GPT-OSS-120B showed adherence to the reference molecules at BF16, Q8, and Q6, but lost the capacity to generate molecules in the desired chemical space at the Q4 quantization level. These results show that quantization can substantially improve the performance of local LLMs for molecular generation without compromising output quality, although model selection and validation is essential for guided exploration of the chemical space.