A Comparison of GEC Tools for Grammatical Error Correction in English

Using the Building Educational Application (BEA) benchmark<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>https://codalab.lisn.upsaclay.fr/competitions/4057, this study compares the capabilities of Google Gemini<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>https://gemini.google.com/, ChatGPT<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup>https://openai.com/chatgpt, DeepSeek<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup>https://www.deepseek.com/en, and the builtin grammar checkers in Google Docs and Microsoft Word for grammatical error correction (GEC). These tools correct a variety of errors, some of which overlap. Based on the BEA benchmark evaluation results, Google Gemini and the Google Docs grammar checker achieve the best <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$F_{0.5}$</tex> scores of 60.2 and 65.86, respectively. Google Docs grammar checker is easy to use and, according to this evaluation, performs well, thus proving to be a viable option for GEC. However, standard grammar checkers are not typically designed for rewriting text to the same extent as GenAI tools; hence, it may be advisable especially for non-native speakers to combine traditional and GenAI grammar correction for the best possible results. However, it is necessary to check the grammatical corrections of LLMs, since generative AI tools suffer from hallucinations, which refers to their tendency to generate information that can be factually incorrect [1].