Challenges in AI: Indexing LGBTQ+ fiction

Efforts to automate cataloging in libraries have progressed significantly, with AI tools like ChatGPT emerging as potential aids. However, automating the subject indexing of LGBTQ+ fiction poses unique challenges. Traditional fiction indexing often overlooks specific themes and characters sought by users, particularly those related to LGBTQ+ identity and issues. Generative Pre-trained Transformers (GPTs) like ChatGPT offer promise in producing detailed subject terms but face biases and inaccuracies. This study explores ChatGPT’s efficacy in generating subject index terms for LGBTQ+ fiction by comparing AI-generated terms with those assigned by professional information specialists in the Queerlit database. The Queerlit database, which uses the QLIT thesaurus for LGBTQ+ terms and general Swedish controlled vocabularies, provides a gold standard for this comparison. Using a sample of 20 full-text works and 20 metadata records from the Queerlit database, ChatGPT was tasked with generating subject index terms. The evaluation revealed that ChatGPT struggled to identify any LGBTQ+ themes, often producing broader and irrelevant terms, even when the index terms were given as input in the metadata. The precision and recall scores were low, highlighting AI’s limitations in this context. The study underscores the need for careful evaluation of AI tools in library and information science and professional practice, particularly for indexing fiction and minority representation. Future research should involve collaboration with both information and subject experts to examine the potential of automatically generated terms that were not previously assigned, as well as to explore the possibility of refining automated indexing methods, and to address inherent biases in AI models.