Reference frames vs re-rolling: the actual toolkit for AI-video continuity
This update was drafted on a schedule by the AI I build with, from real project notes — part of the vibecoding experiment this blog documents.
When I wrote that continuity is the wall, I listed the four patches you can bolt on to fake persistence between shots and then waved at them in one sentence — "the seams are visible, the drift creeps back." That's true but it's not useful. If you're actually trying to get two shots of the same thing out of a model today, you need to know which patch to reach for, what it actually buys, and what it costs you to use it.
So here's the hands-on version, from running Cadence. The stack is Higgsfield with Seedance 2.0 underneath, 8-second 9:16 clips. The four reels on the project page — AURA, AXIOM, LUMEN, STRIDE — are each a single generation, so honestly they dodged this problem. The moment you want shot two of the same can, you're in it. These are the tools.
First, the thing you're fighting
Every generation is an independent draw. The model isn't holding a scene and showing you another angle of it — it's painting a brand-new scene each time, from scratch, from a description. "Re-rolling" — just generating again and hoping it matches — doesn't converge on consistency, because there's nothing pulling the two draws toward each other. Every patch below is a different way of forcing information from generation one into generation two.
Patch 1: reference image
What it is. You feed a still — a frame from your last clip, or a product photo — as a visual anchor alongside the prompt. The model tries to honor it.
What it buys. Coarse identity. The can is roughly the right can, the color's right, the general vibe carries. Good for "same product, totally different shot" where you need the viewer to believe it's the same brand, not the same physical object.
What it costs. Precision. A reference image pins look, not detail — the label text will re-hallucinate, small geometry shifts, logos warp. And the more strongly you weight the reference, the more the model's composition collapses toward the reference's composition, so you lose framing freedom. Push it too hard and every shot looks like a slightly-melted copy of the first one.
When I use it. Establishing shots, or any cut where the two shots are far enough apart in angle and time that the audience isn't doing a frame-by-frame comparison.
Patch 2: last-frame conditioning
What it is. You take the final frame of clip A and hand it to clip B as its starting frame. The new generation begins exactly where the old one ended and animates forward.
What it buys. A genuinely seamless cut at one specific instant — the join. For a continuous move (push-in that carries across a "cut," a hand that keeps traveling) this is the strongest tool there is, because the first frame isn't approximated, it's identical.
What it costs. It decays. The match is perfect at frame one and drifts every frame after — by second three of the new clip you're back to cousins, because once the model is animating it's re-sampling again and the anchor's influence fades. So it buys you a clean transition and then slowly gives the problem back. It also chains badly: clip C conditioned on clip B's drifted last frame inherits the drift and adds more.
When I use it. Match-on-action moments and any place I want an invisible join. Never for holding identity across a whole sequence.
Patch 3: video-to-video
What it is. An existing clip constrains the new generation — its motion, layout, and timing get carried over while you restyle or alter the content.
What it buys. Structural continuity. Camera move, blocking, pacing, and screen direction stay locked because they're inherited from a real clip instead of re-invented. This is the one that fixes the 180-degree-line coin flip — if the source has her on the left, the output keeps her on the left.
What it costs. You need a source clip that already has the motion you want, which is a chicken-and-egg problem — often I generate a rough one just to drive the real one, which is two generations for one shot. And the tighter you bind to the source, the less the model is free to make the output good; loosen it and the constraints leak. It's the highest-effort patch by a distance.
When I use it. When screen direction or a specific camera move absolutely has to survive a cut and reference frames aren't holding it.
Patch 4: a character model
What it is. You train a small model on one subject — a face, sometimes a product — and then every generation can invoke it.
What it buys. The best identity persistence available, and the only one that's truly reusable — once it's trained, every future shot in every future clip can call the same face without re-feeding references.
What it costs. Setup time and a data requirement (you need enough clean images of the subject, which for a client product you often don't have). It only covers the subject — it does nothing for the room, the light, the wardrobe, the props. And there's a quality tax: heavily-invoked character models tend to flatten expression and lighting range, because the model is now pulling hard toward its training distribution.
When I use it. Anything with a recurring human face across multiple clips. Not worth it for a one-off.
How they stack
In practice a single sequence uses several at once: character model for the face, last-frame conditioning at each join, a reference image for the environment, video-to-video on the one shot where the camera move matters. Each patch is smuggling a different slice of state between two stateless processes, and they don't compose cleanly — every one you add costs a little more of the model's freedom to just make a good-looking shot, so the output gets more consistent and less alive at the same time.
That's the trade that defines this whole phase of the tech. You can have continuity or you can have the model's full range, and every tool above is a dial between them. None of it is the real fix — the real fix is a persistent scene the model edits instead of re-samples, and that doesn't exist yet in anything production-ready.
I don't have paid client motion work to point at — the studio's live, the reels are demos, nobody's hired me for a multi-shot spot. So this is a report from the build, not from a delivery. But if you're trying to do it right now, this is the kit, and the honest summary is: you can get maybe two or three shots to hang together with real effort, and the fourth one is where the drift wins.
This one's auto-drafted from my notes on a schedule. If a number isn't in the notes, it doesn't show up here — I'd rather leave a blank than make something up.