Bidirectional Screen Sharing for Generative Artificial Intelligence: A Unified Framework for Real-Time Visual Collaboration Between Users and AI Systems (BSS-GAI)

Screen sharing has become an established mechanism for visual communication, remote assistance, collaborative work, and interactive computing. With the development of generative artificial intelligence (AI), screen-based interaction has expanded beyond conventional human-to-human communication to include systems capable of interpreting user interfaces, analyzing visual information, and operating digital environments. Existing approaches primarily emphasize the transmission of a user's screen to an AI system for visual understanding or the presentation of an AI agent's computer environment to a human observer. However, these capabilities are frequently implemented as separate interaction mechanisms rather than as components of a unified, direction-independent screen-sharing architecture. This paper proposes Bidirectional Screen Sharing for Generative Artificial Intelligence (BSS-GAI), a conceptual framework that enables both users and generative AI systems to initiate, receive, manage, and dynamically exchange screen-sharing sessions. The framework defines four principal interaction configurations: user-to-AI sharing, AI-to-user sharing, simultaneous bidirectional sharing, and dynamically switchable directional sharing. It further introduces shared-screen visualization, independent stream management, session synchronization, selective display-surface transmission, contextual screen annotation, permission negotiation, temporary authorization, and human-controlled interruption. Unlike conventional screen-sharing mechanisms that treat the human participant as the primary source of visual information, BSS-GAI conceptualizes the user and AI system as equivalent logical participants in a shared visual communication environment, while preserving their different technical capabilities and authorization responsibilities. The proposed architecture separates screen acquisition, transmission, interpretation, presentation, and interaction control, allowing the framework to operate across desktop computers, mobile devices, virtual environments, and AI-controlled computing environments. Security, privacy, consent, and the potential risks associated with exposing sensitive screen information are treated as fundamental architectural requirements. This paper establishes a research foundation, formalizes the interaction model, identifies technical requirements, and proposes an evaluation methodology for future implementations. BSS-GAI provides a conceptual basis for transforming conventional one-directional AI screen interaction into a unified visual collaboration paradigm between humans and generative AI systems.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-16
DOI
https://doi.org/10.5281/zenodo.24278147
Primary Topic
Cloud Computing and Remote Desktop Technologies
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Bidirectional Screen Sharing for Generative Artificial Intelligence: A Unified Framework for Real-Time Visual Collaboration Between Users and AI Systems (BSS-GAI)

Jincheng Zhang
6 links
Zenodo (CERN European Organization for Nuclear Research)
Cloud Computing and Remote Desktop Technologies
preprint

Bidirectional Screen Sharing for Generative Artificial Intelligence: A Unified Framework for Real-Time Visual Collaboration Between Users and AI Systems (BSS-GAI)

Jincheng Zhang
preprint en

Abstract

Screen sharing has become an established mechanism for visual communication, remote assistance, collaborative work, and interactive computing. With the development of generative artificial intelligence (AI), screen-based interaction has expanded beyond conventional human-to-human communication to include systems capable of interpreting user interfaces, analyzing visual information, and operating digital environments. Existing approaches primarily emphasize the transmission of a user's screen to an AI system for visual understanding or the presentation of an AI agent's computer environment to a human observer. However, these capabilities are frequently implemented as separate interaction mechanisms rather than as components of a unified, direction-independent screen-sharing architecture. This paper proposes Bidirectional Screen Sharing for Generative Artificial Intelligence (BSS-GAI), a conceptual framework that enables both users and generative AI systems to initiate, receive, manage, and dynamically exchange screen-sharing sessions. The framework defines four principal interaction configurations: user-to-AI sharing, AI-to-user sharing, simultaneous bidirectional sharing, and dynamically switchable directional sharing. It further introduces shared-screen visualization, independent stream management, session synchronization, selective display-surface transmission, contextual screen annotation, permission negotiation, temporary authorization, and human-controlled interruption. Unlike conventional screen-sharing mechanisms that treat the human participant as the primary source of visual information, BSS-GAI conceptualizes the user and AI system as equivalent logical participants in a shared visual communication environment, while preserving their different technical capabilities and authorization responsibilities. The proposed architecture separates screen acquisition, transmission, interpretation, presentation, and interaction control, allowing the framework to operate across desktop computers, mobile devices, virtual environments, and AI-controlled computing environments. Security, privacy, consent, and the potential risks associated with exposing sensitive screen information are treated as fundamental architectural requirements. This paper establishes a research foundation, formalizes the interaction model, identifies technical requirements, and proposes an evaluation methodology for future implementations. BSS-GAI provides a conceptual basis for transforming conventional one-directional AI screen interaction into a unified visual collaboration paradigm between humans and generative AI systems.

Zenodo (CERN European Organization for Nuclear Research)
Cloud Computing and Remote Desktop Technologies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon