Mindstorms in Natural Language-Based Societies of Mind
Inspired by Minsky's Society of Mind, Schmidhuber's Learning to Think, and other more recent works, this paper proposes and advocates for the concept of natural language-based societies of mind (NLSOMs). We imagine these societies as consisting of a collection of multimodal neural networks, including large language models, which engage in a “mindstorm” to solve problems using a shared natural language interface. Here, we work to identify and discuss key questions about the social structure, governance, and economic principles for NLSOMs, emphasizing their impact on the future of AI. Our demonstrations with NLSOMs-which feature up to 129 agents-show their effectiveness in various tasks, including visual question answering, image captioning, and prompt generation for text-to-image synthesis.
Authors
- Kazuki Irie
- Louis Kirsch (ORCID: https://orcid.org/0009-0001-5203-4144)
- Francesco Faccio
- Dylan R. Ashley (ORCID: https://orcid.org/0000-0001-6148-8802)
- Piotr Piękos
- Jinjie Mai (ORCID: https://orcid.org/0000-0002-3396-1970)
- Vincent Herrmann
- Imanol Schlag
- Bernard Ghanem (ORCID: https://orcid.org/0000-0002-5534-587X)
- Hasan Abed Al Kader Hammoud (ORCID: https://orcid.org/0000-0002-2991-0804)
- Mingchen Zhuge (ORCID: https://orcid.org/0000-0003-2561-7712)
- Abdullah Hamdi (ORCID: https://orcid.org/0000-0003-3989-7540)
- Weimin Shi (ORCID: https://orcid.org/0009-0002-2751-3204)
- Róbert Csordás
- Deng-Ping Fan (ORCID: https://orcid.org/0000-0002-5245-7518)
- Jürgen Schmidhuber
- Aleksandar Stanić (ORCID: https://orcid.org/0000-0002-6399-7536)
- Yuhui Wang (ORCID: https://orcid.org/0000-0002-0502-7486)
- Wenyi Wang (ORCID: https://orcid.org/0000-0003-1619-0294)
- Bing Li (ORCID: https://orcid.org/0000-0003-4987-6129)
- Shuming Liu (ORCID: https://orcid.org/0000-0002-4949-4318)
- Guohao Li (ORCID: https://orcid.org/0000-0003-0260-5129)
- Aditya A. Ramesh
- Haozhe Liu (ORCID: https://orcid.org/0000-0003-1191-3529)
- Mengmeng Xu
- Anand Gopalakrishnan
Institutions
- Nankai University (CN)
- Dalle Molle Institute for Artificial Intelligence Research (CH)
- Harvard University Press (US)
- ETH Zurich (CH)
- University of Oxford (GB)
- Science Oxford (GB)
- King Abdullah University of Science and Technology (SA)
- Beihang University (CN)
- Stanford University (US)
Publication Details
- Journal
- Computational Visual Media
- Published
- 2025-02-01
- DOI
- https://doi.org/10.26599/cvm.2025.9450460
- Citations
- 4
- Primary Topic
- Cognitive Science and Mapping
- Type
- article
- Field-Weighted Citation Impact
- 7.44