Task-Based Evaluation of a GPT-4 ‘Dewey Decoder’ Tool for Dewey Decimal Classification in Academic Libraries in Sri Lanka

This study evaluates an AI-powered Dewey Decimal Classification assistant the Dewey Decoder, a custom tool built on OpenAI GPT-4 for its effectiveness in academic libraries. A task-based experiment involved 61 purposively selected Sri Lankan university librarians who classified sample resources using the Dewey Decoder and their normal manual workflow. Data were gathered on (a) accuracy (agreement with an expert gold standard), (b) efficiency (time per classification task), and (c) usability (5-point Likert survey and open-ended feedback). Results show the Dewey Decoder achieved a mean accuracy rating of 4.32 / 5, correctly identifying broad classes in 93 % of cases while revealing occasional errors with nuanced or culturally specific works. Eighty-five per cent of participants reported time savings; 69 % completed each classification in under three minutes, compared with over five minutes manually. Usability was rated 4.52 / 5, with participants praising the tool’s step-by-step guidance but noting limits on Sinhala/Tamil support and the five-query cap in free GPT-4 accounts. Although purposive sampling ensured expert input, it constrains generalisability beyond similar academic settings. Overall, findings indicate that GPT-4-driven assistants can substantially enhance cataloguing speed and consistency, provided language coverage and integration with library systems are improved. Future research should test the tool across more diverse collections and librarian populations to validate these gains.