Using Five LLMs to Suggest Factors Related to the Extent of Success in IS Usage

This exploratory paper pursues a new direction in IS usage research. It starts with preliminary research that uses ChatGPT-4o in a four-phase “refining” process to produce an initial version of a new theory of IS/IT acceptance and IS success. The main section of the paper uses LLMs (ChatGPT-4o, Claude Sonnet 3.7, Grok 3, Llama 3.1, and Perplexity) to produce organized lists of success factors for IS usage. Each of the five LLMs responded to the same four prompts that requested increasing degrees of specificity in those success factors. An additional prompt instructed each of the five LLMs to refine results from the other four LLMs down to a new list of 15 or more success factors for IS usage. The conclusion summarizes methodological contributions and knowledge contributions.