GenAI Empowered Script to Storyboard Generator

The research presents an innovative solution for automating the generation of storyboards from screenplays through the integration of advanced AI technologies. By taking a screenplay as input, the system utilizes cutting-edge neural networks to recognize characters and objects, enhancing scene comprehension. A refined Bi LSTM model is employed to extract the nuanced emotional tones embedded within in dialogues in each scene, providing valuable insights into character dynamics and narrative depth. Through the application of regular expressions, key scene attributes such as time, place, and location are extracted to establish contextual relevance. A Facebook BART-large-CNN model is then employed to generate concise summaries of each scene, enhancing comprehension efficiency. Through script summarization and tag extraction from the Movie Plot Synopses with Tags dataset, a smooth transition between scenes is enabled. These extracted features are structured into a prompt using a rule based approach, facilitating seamless integration into the subsequent creative phase. Finally, a stable diffusion model is employed to generate scene-by-scene coherent storyboard, incorporating all extracted elements to streamline the visual storytelling process where coherency is achieved with the help of cosine similarity between prompts. This comprehensive approach not only automates tedious tasks but also enhances creativity and efficiency in storyboard creation.