Shaduf.Research preview
Real-Time AI Video/Live Systems

Genie 4: live world control, few-minute sessions

Google says: its audiovisual world responds to text and camera input, but access is a trusted-tester preview and continuous interaction lasts only a few minutes. The published examples are shortened; price and viewer delay are not public. Check the access and session limit.

Hy-Nomas: a 4090 fit claim, an eight-B200 browser test

Project and community reports: the low-memory RTX 4090 preset has no published viewer rate. The eight-B200 Reactor route generated 18.53 FPS over two minutes; a separate 4090 ComfyUI job took 85 seconds for 10 seconds of 1280×704 video. Decoder-weight rights for paid use need clarification. Compare the routes.

SGF+: a 24-hour paper result, a four-minute browser default, and a slower 5090 file route

Project reports: a B200 Reactor service generates about 19 FPS for 16-FPS browser playback; prompt edits wait for the next chunk and buffered frames. An independent ComfyUI pack needs 56.6 warm seconds for a 20-second 5090 output. Neither establishes a 24-hour viewer session or full cost. Compare the three routes.

Micro-World 2: 42.49 decoded FPS on one MI350X, not a priced viewer session

AMD reports: its 352×640 action-controlled model reaches 42.49 decoded FPS with a lightweight decoder or 32.12 with the original Wan VAE. The faster decoder reduces visual quality; no browser response, accepted long session or full bill is measured. Read the route and decoder check.

SolarWM-H3: 1.15 seconds from keypress to screen, on eight Blackwell GPUs

Project reports: its browser demo reaches a 1.15-second median for a matching frame after a keypress. The integrated backend reports 24.38 steady-state FPS. The documented route is video-only and needs eight Blackwell GPUs; no p95, long-session quality or complete bill is published. Read the browser and hardware check.

EdenLive on a 5090: 25.17 FPS in a file, not yet a browser result

Project reports: LightX2V’s warm 717-frame, 1280×704 proxy-video job runs at 25.17 inference FPS on one RTX 5090. It saves an MP4. EdenLive’s WebRTC service guide lists an 80-GB GPU in its verified environment; action-to-screen delay and a full bill remain unmeasured. Check the two deployment paths.

One RTX 5090: 24-fps H3 playout, not measured viewer control

Community project reports: FastH3 Live v2.1.0 makes each 15.08-second 448×448 video-and-audio clip in 10.8 warm seconds on a pinned TaoMate, Light VAE and attention stack. The prefilled channel plays queued scenes; no viewer prompt-to-visible timing or full bill is reported. H3 derivative-weight rights have territorial limits. Read the setup and test boundary.

TaoMate-H3: merged video, not yet synchronized AV

Project says: vLLM-Omni merged a one-session WebSocket route on 19 October. The two-H200-class FP8 test reports a 4.74-second median per 4.958-second window, but that is not viewer response. The merged MP4 carries video only; AAC work remains open. Check the route and limits.

Latest check: ComfyStreamerH3 on one RTX 5090

Project reports: two 15.08-second 448×256 clips in a 23.51-second warm paired job. Its HLS browser waits for two completed clips and queues a new prompt until the current clip ends. The viewer response is unmeasured; the managed example uses a different GPU and price. Read the two-clock check.

StreamForce: force control with a browser receiver

The released image-to-video demo takes point or wind changes while generating. On one H200 the newer implementation reports about 23 generated FPS, while 16-FPS playback, a local buffer test and short visual coherence set separate limits. See the measured queue trade-off.

Latest check: Waypoint-1.5 browser route

Project reports: one H100 generated 82 FPS and one A10G 25 FPS at 720p from an instance-local client. A separate remote sample delivered 44.6 FPS. The browser route also waited 645 seconds on cold compilation and visibly lost scene coherence after roughly 512 generated frames. Compare the clocks and limits.

TeleFuser: one minute of media, 79-second session

Project report: a four-H100 LingBot-World v2 LiveKit replay generated 59.75 seconds of media; first client frame arrived at 5.9822 seconds and the session lasted 79.2523 seconds. Target compute ran above 16 FPS, but the receiver test does not prove an uninterrupted live hour, accepted quality, control delay or complete cost. Read the two-clock check.

Scope: a browser route with a paid-show gate

Project says: its WebRTC guide sends source video, returns generated video and accepts live prompt changes. Community report: one remote-beta H100 event test took about ten minutes to cold-ready; LongLive T2V and StreamDiffusionV2 V2V timings are not interchangeable. The repository alpha license is non-commercial, and event delay, one-hour rollover and full cost remain open. Read the show-readiness check.

Hallo-Live: streaming model, saved-file reference

Authors report: 20.38 generated FPS and 0.94-second model latency on two H200s for text-to-audio-video. The inspected public command saves a 24-fps MP4, above the paper throughput; a viewer-visible WebRTC/RTMP session and interruption path are not documented there. Read the route check.

LiveAvatar is a narrow live research route

Its paper reports faster-than-playback audio-driven avatar generation on five H100s, but also estimates roughly three seconds end to end with transport. The official release still lists easy interactive-stream UI and TTS integration as unfinished. A delivered, interruptible viewer session has not been verified here. Read the route and clock check.

Live systems · checked 13 October 2026

A continuous stream needs a recovery plan for late clips

A smooth demo can hide a remote API wait, a queue, an idle filler, a repeated frame, a slower playback rate, a hard scene cut, or a vote that waits offscreen. Ask what continues when the next clip is late, a prompt fails, a vote closes, or a session ends.

Architecture before model

  1. Input: accept prompts, proposals, votes, or controls; decide moderation and ordering.
  2. Selection: state who decides the next action, when that window closes, and what happens to late or losing input.
  3. Generation: measure acceptance-to-ready time, not an advertised frame rate.
  4. Deployment: state the client host, inference host, task, adapter, server revision, GPU/host memory, and HTTP response boundary.
  5. Buffer: hold enough finished or in-flight playable work for a slow render or retry.
  6. Playout: choose the output fps; state filler, transitions, fallback footage, and what black, silence, a frozen frame, or slowed motion means.
  7. Delivery and recovery: timestamp the visible result, reconnect sessions, preserve story state if possible, and log the gap.

StreamRender-H3 has a browser feedback loop

Project says: a racing engine sends semantic frames into H3 and receives rendered frames on eight GB200s. Its 218 ms render step is not control-to-screen latency. The asset card reports about 1.03-second browser publication intervals. One racing scenario is validated; another game, a paid session and a full cost are not. See the three-clock check.

Griffin-Lite: conversation research, not customer access

Provider says: Tavus's selected-tester preview watches and listens while generating a face and voice. Its 0.43-second audio-to-video generator result is not the same as NVIDIA VideoFDB's 1,892/2,232 ms median generation/perception event timing. Customer access and Griffin-specific pricing are not published. Compare the clocks.

Vidu S2-Editing has a live transport contract

Provider says: an operator publishes video into an RTC session, receives an edited RTC stream, and can relay it to RTMP. A control signal may change the reference or scenario mid-session, but a successful switch has no separate acknowledgment. Measure the first changed receiver frame, not just the signal response. The provider also documents that relay failure ends the live session. See the cost and test boundary.

Workflow scope comes before a continuity design

Project says: FastVideo’s new 8-Step V2 schedule is T2AV-only. ComfyUI says: its current template for the same checkpoint also permits first/last-frame FL2VA. Until that conflict is resolved, do not assume a local FastH3 clip can begin where the last one ended. A buffer cannot repair a missing conditioning contract.

A carried frame is another boundary, not a visible transition

Project says: FastVideo’s Dreamverse FastH3 Preview v1 profile feeds each segment’s last frame into the next segment as first-frame conditioning. The documented profile uses four visible GPUs and 124-frame 768×1344 segments.

This is stronger evidence of an intended continuity mechanism than a bare queue. It still does not say whether the next segment arrives in time, visibly respects the frame, survives a retry, or keeps audio and story coherent. The README’s cold compile warm-up can take tens of minutes, so log boot-to-ready before timing a session.

A complete MP4 can beat playback while the screen still waits

Project says: vLLM-Omni’s maintainer report returned a complete 10.125-second 1344×768 / 24 fps FastH3 T2VA MP4 in 8.678 and 8.710 seconds on a warmed, dedicated eight-B300 profile. Its timer begins with synchronous submission and ends when the whole MP4 returns.

That is a meaningful complete-response threshold. The report explicitly does not call it streaming delivery or time to first frame. It excludes startup, compilation, and an excluded warmup; its raw bundle is pending. It does not reveal queue time, concurrent capacity, browser decode, media delivery, retries, or what a viewer sees while waiting.

The earlier vLLM-Omni full-H3 FL2VA row remains a different route: four B300 GPUs, 209 frames at 1248×768, and 86.964 seconds mean HTTP client latency for about 8.71 seconds of output. Do not use the FastH3 T2VA number to sell a first/last-frame or live-continuity workflow.

Local capacity still leaves a long clip wait

Project says: vLLM-Omni completed one full-H3 50-step T2VA job on two RTX 5090s: 124 1344×768 / 24 fps frames in 8 minutes 38 seconds of client end-to-end time. The route needs at least 200 GiB available host RAM, with 384 GiB recommended.

That establishes a documented offline capacity path. It does not state when a first frame becomes visible, how a second request queues, whether a first/last-frame task works on the same hardware, or what a viewer sees while the renderer works. Do not use the lower GPU count as a live-system claim.

A fast next clip does not reveal the control-loop wait

Provider says: Yoroll’s YoLive lets a group propose and vote on the next scene. Its launch announcement says H3 Superfast generates 10 seconds of 768p, 24 fps video with native audio in four seconds on eight B200 GPUs. That figure leaves a six-second gross lead over the cited clip duration.

The missing timestamps are proposal close, moderation, winning instruction, generator acceptance, first delivered frame, and visible story turn. Without them, the source supports a faster-than-playback clip claim and a collective next-scene product—not a measured audience response time.

A local channel can close the timing gap by retiming

Community project says: FastH3 Live keeps two prompts queued on one RTX 5090 while it streams a prior clip. Its current dataset card says the named 448×448 route sustains 22.1 fps. The source explains that it retimes the audio and 362 source frames to the pace it can deliver; its earlier 18-fps example is 75% native motion.

This is a valid system design for an unattended channel if its visual format makes the motion compromise acceptable. It is not evidence that a viewer’s command changes the active scene, that 24-fps motion holds, or that the route survives a bad generation without a visible wait.

Open session, group choice, remote service, or buffered clips

Provider says: H3 Max Director uses an open WebRTC session, with new directions sent over a data channel while output tracks play. Provider says: YoLive is a group-selected next-scene route. Project says: vLLM-Omni is a separate H3 service route. Community project says: a local FastH3 channel can keep distinct prepared scenes moving at a source-stated sub-24-fps pace.

These are four different promises. Neither one implies the other three.

Nine questions before you call it live

  • Who proposes or directs the change, and who selects it?
  • When does that control window close?
  • Where does the request run, and when does the inference server accept it?
  • How long from a new direction to playable media and then visible change?
  • What are the source fps, delivered fps, and visible motion trade-off?
  • Which GPU profile, checkpoint, adapter, frame-conditioning asset, server state, and warm-up plan produced the next ready clip?
  • How many ready or in-flight clips sit ahead of playback?
  • What appears after a miss or a failed render?
  • Does the next clip preserve the scene, voice, and viewer’s choice—or begin a new one?

Search published pools, pages, reports, and evidence.