A Provider-Agnostic Protocol and Observational Pilot for Reproducible Evaluation of Browser-Based AI Video Generators

Browser-based AI video generators are difficult to evaluate reproducibly because interfaces, model routing, and outputs change rapidly. This work presents a provider-agnostic protocol for evaluating accessibility, generation controls, provenance, technical media properties, perceptual quality, prompt alignment, temporal consistency, and operational reliability. An observational pilot analyzes 16 publicly displayed showcase videos (four per service) from Flow AI Video, Seed Imagine, Dola AI, and Luma Dream Machine App. The accompanying reproducibility package includes the experiment manifest, audit script, machine-readable results, and summary. The pilot is descriptive and does not claim a controlled ranking of underlying models. The author is affiliated with the four evaluated web properties; this potential commercial conflict is disclosed and bounded through open methods, machine-readable results, and non-ranking claims.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-24
DOI
https://doi.org/10.5281/zenodo.22934013
Primary Topic
Scientific Computing and Data Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
preprint

A Provider-Agnostic Protocol and Observational Pilot for Reproducible Evaluation of Browser-Based AI Video Generators

LISA
Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
preprint

A Provider-Agnostic Protocol and Observational Pilot for Reproducible Evaluation of Browser-Based AI Video Generators

LISA
preprint en

Abstract

Browser-based AI video generators are difficult to evaluate reproducibly because interfaces, model routing, and outputs change rapidly. This work presents a provider-agnostic protocol for evaluating accessibility, generation controls, provenance, technical media properties, perceptual quality, prompt alignment, temporal consistency, and operational reliability. An observational pilot analyzes 16 publicly displayed showcase videos (four per service) from Flow AI Video, Seed Imagine, Dola AI, and Luma Dream Machine App. The accompanying reproducibility package includes the experiment manifest, audit script, machine-readable results, and summary. The pilot is descriptive and does not claim a controlled ranking of underlying models. The author is affiliated with the four evaluated web properties; this potential commercial conflict is disclosed and bounded through open methods, machine-readable results, and non-ranking claims.

Zenodo (CERN European Organization for Nuclear Research)
Scientific Computing and Data Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.