3D Scene Reconstruction for Real-Time Video Editing. A Performance Study Conducted on Web Browsers

Multimedia processing has become increasingly critical in Web applications, necessitating advanced machine learning tools for video analysis within the browser. Current front-end Web technologies, such as JavaScript and WebGL, empower complex tasks like object recognition, depth estimation, and 3D scene reconstruction directly within the Web browser. This approach places a strong emphasis on optimizing real-time performance and front-end responsiveness. The research presents a pipeline that recognizes objects in video footage and replaces them with 3D models, enabling the reconstruction of entire scenes as complete 3D environments. A fully reconstructed 3D environment facilitates more immersive and dynamic experiences in virtual/augmented reality (VR/AR). Furthermore, to assess performance, extensive benchmarking has been conducted on various Web browsers: Google Chrome, Mozilla Firefox, and Apple Safari. The study evaluates processing speed, frame rendering time, memory consumption, and display delays. Additionally, it investigates the impact of Web browser differences on performance. Furthermore, this work investigates methods to enhance speed and efficiency.