Supporting User’s Cognitive Ability as the Key Agenda in Multimodal LLM/GenAI R&D

While Multimodal LLM and Generative AI are drawing much attention due to its immense potential for transforming every aspect of our lives, the characteristics of the technology and its promises almost always imply reduced cognitive efforts by its end users. This can be compared to our not-so-smart but still very powerful and convenient day-to-day technologies we are used to today, afforded by ever-improving computational power, availability of smartphones and the internet: our phone apps and web services are efficient, accurate and convenient, helping us save our mental efforts (e.g. memorising, calculating, summarising, reflecting, etc.). There are growing amount of scientific evidences on how the prolonged use of and reliance to these apps and services undermines our natural cognitive abilities - because almost by definition these tools are there to help their users bypass the cognitive efforts - and the Human-Computer Interaction (HCI) and UI/UX communities are starting to address this to amend and extend the design knowledge (principles, usability guidelines, heuristics, etc.) to take this into account. Is it possible to re-design our apps and services in such a way as to keep our cognitive abilities active while at the same time help us achieve the tasks that those tools were designed for in the first place? This talk will point out how our future applications of Multimodal LLM and GenAI will have similar impact to people’s cognitions as the conventional apps and services have done so far, and what our stance as the researchers in the field could or should be to minimise such negative consequences. This has implications to the agenda for research directions for those studying and developing Multimodal LLM and other Generative AI technologies.