Shaduf.Research preview
Real-Time AI Video/StreamEval: visual quality does not ensure control

Pool topic: Real-time, continuous, and interactive AI video generation. Question revision: 1. Exact question: What can generate continuous, interactive AI video today, and which setups work at which cost, latency, hardware requirements, and quality?

Checked 2 November 2026

StreamEval: visual quality did not ensure action following

Benchmark authors report: In StreamEval, HappyOyster 1.0 Adventure scored 58.34 for visual Quality, the highest in its world-generation table, but 5.56 for Following. Krea Realtime led Following at 25.93 with a lower Quality score of 35.61. These are scores on scripted 15–300-second sessions, not live-user satisfaction or network latency.

What the scores test

StreamEval sends new prompts and camera controls during a session. Following checks whether the requested operation appears on the intended target. Continuity checks motion, camera behavior, object permanence and spatial relations. The paper reports 17 world-generation configurations within a larger five-task, 300-case evaluation. It ranks text- and image-conditioned configurations in one table and shows the text result for a model evaluated both ways. Definitions and test cases are published in the project repository.

MemFlow leads the four-metric world Overall at 33.36. That does not make it the fastest or cheapest service. The paper uses automated visual judges and authored scripts; we did not reproduce the evaluation. It does not supply a viewer preference test, a route-specific delivered-hour bill or an end-to-end latency comparison for these systems.

What to do before choosing a route

Do not select a world model from a polished clip alone. If you can access a route, record one timed action, the first visible response, an unchanged scene element and what happens after a failed action. Keep the model version, client, GPU, playback rate and full bill with that trace. If access is restricted, treat the leaderboard as a reason to ask for that evidence, not as a deployment ranking.

Search published pools, pages, reports, and evidence.