GEEToker: you can boost LLMs’ JavaScript-based geospatial code generation for Google Earth Engine with a model-agnostic tokenizer

As a platform that integrates large-scale remote sensing data with cloud computing capabilities, Google Earth Engine (GEE) has been widely applied to tasks such as land cover monitoring and environmental assessment. However, its code development process requires strong programming skills and geospatial expertise, resulting in a high entry barrier for many users. Although existing large language models (LLMs) demonstrate considerable potential for code generation, their performance in GEE code generation tasks is often constrained by the limitations of general-purpose tokenizers. These tokenizers tend to produce inaccurate semantic segmentation of domain-specific syntax, leading to generation hallucinations during training and inference. To address this issue, this study proposes GEEToker, a model-agnostic and transferable tokenizer construction framework tailored for GEE code generation. The framework achieves structured optimization of GEE code through three core modules – data normalization, seed token mining and vocabulary enhancement training – and introduces several key datasets, including GEE-PT, GEE-SFT, GEEtokenization-Bench and GEE-Bench. Experimental results show that GEEToker reduces token consumption by 46.47%–53.78%, increases token coverage by 2.08%–17.82%, and shortens training and inference time by 4.20%–28.84%. Furthermore, it improves performance in entity extraction, code generation and code completion tasks by 2.02%–20.00%. These results validate the effectiveness of tokenizer optimization in enhancing LLMs’ geospatial code generation capabilities and suggest that, in resource-constrained or time-sensitive environments, prioritizing tokenizer training and replacement represents a highly cost-effective strategy. This work offers a new perspective and valuable insights for advancing LLM applications and tokenization research in vertical domains.