Components of Character: Exploring the Computational Similarity of Austen's Characters

Understanding characters is a key topic of interest in computational explorations of literary fiction.There are a growing number of tools available for extracting information about characters from text.Recent work proposes AustenAlike, a three-part benchmark for Jane Austen character similarity, to evaluate computational representations of character.I extend this exploration of feature-based representations of character with new experiments, including representations build from multiple features.I also explore a way of obtaining vector representations of characters using a large language model by generating short textual descriptions of characters using GPT-4, and then taking their embeddings.My results show surprisingly little advantage from combining multiple kinds of features, suggesting that the different features capture similar information to each other.I find that the LLM-generated textual descriptions perform best on expert benchmark, outperforming direct elicitation of character similarity judgments from the same model.However, the expert benchmark remains challenging across all representations explored, showing that there is much room for improvement in capturing the more nuanced aspects of character similarity that human readers identify.