Reproducibility and quality of hypertrophy-related training plans generated by GPT-4 and Google Gemini as evaluated by coaching experts

Large Language Models (LLMs) are increasingly utilized in various domains, including the generation of training plans. However, the reproducibility and quality of training plans produced by different LLMs have not been studied extensively. This study aims to: i) investigate and compare the quality of muscle hypertrophy-related resistance training (RT) plans generated by Google Gemini (GG) and GPT-4, and ii) the reproducibility of the RT plans when the same prompts are provided multiple times concomitantly. Two distinct prompts were used, one providing little information about the training plan requirements and the other providing detailed information. These prompts were input into GG and GPT-4 by two different individuals, resulting in the generation of eight RT plans. These plans were evaluated by 12 coaching experts using a 5-point Likert scale, based on quality criteria derived from the literature. The results indicated a high degree of reproducibility, as indicated by coaching expert evaluation, when the same distinct prompts were provided multiple times to the LLMs of interest, with 27 out of 28 items showing no differences (p > 0.05). Overall, GPT-4 was rated higher on several aspects of RT quality criteria (p = 0.000-0.043). Additionally, compared to little information, higher information density within the prompts resulted in higher rated RT quality (p = 0.000-0.037). Our findings show that RT plans can be generated reproducibly with the same quality when using the same prompts. Furthermore, quality improves with more detailed input, and GPT-4 outperformed GG in generating higherquality plans. These results suggest that detailed information input is crucial for LLM performance.