A BLEU Perspective on Text Generation: Markov vs. LSTM Models in Storytelling

Natural language generation is one of the most pivotal factors that bridges the gap between creativity and intelligence between humans and automated machines that can act like humans. It is the driving agent for many applications ranging from simple chatbots to complex models like OpenAI’s ChatGPT. This paper explores two methods for natural language generation for storytelling; one being a probabilistic approach using Markov models and a deep learning approach that uses Long Short-Term Memory (LSTM) networks. The objective of this paper is to generate well-articulated, meaningful random stories using seed text and a n-gram specification. The language generation model is trained on the Sherlock Holmes stories dataset. We measure their ability to create engaging stories and compare the results using BLEU scores. The findings offer insights into the potential of blending statistical models with modern deep learning techniques but also highlights the limitations of both models due to their varying complexities. While the Markov models provide substantially diverse and meaningful stories, the LSTM generates excellent outputs but with certain drawbacks with limited inputs and large runtimes.