Xianzi Insights

We Built a Scroll-Driven Frame-Sequence System — and Open-Sourced It for Free

林政賢 · 2026-08-13

When you scroll down the tangyi.mx homepage right now, the background moves with you. Stop, and it freezes on that frame; scroll back up, and it runs in reverse. That is not a video.

We open-sourced the homepage's "scroll-driven background"

It's a sequence of frames driven frame-by-frame by code. To make it we wrote scroll-frame-sequence: a 535-line, zero-dependency JavaScript library. It relies on no external packages, splits a video into a frame sequence and drives it precisely from the scroll position, bypassing every browser restriction on video autoplay and scroll sync.

The reason we built it is simple: when making complex interactive video pages, we couldn't stand the stutter of native video under frequent scrolling. Once a page drops frames, the experience collapses. The system wraps the asset tooling into four Python commands that output spritesheets for any resolution. If you're building highly interactive pages, take a look at our AI demos to see how this architecture stays smooth on mobile — it isn't only about performance, it's about the animation looking the same on every device.

How does a scroll animation avoid crashing a phone's memory?

You think an image is just an image, but to the browser every decoded image is competing for memory. This was the biggest hole we fell into: a phone's rendering limit is brutal, and once you cross it the page crashes outright or freezes.

The memory a browser uses to decode an image follows a fixed formula: width × height × 4 bytes. So even if the spritesheet file itself is small, the decoded size can instantly blow past what the device has.

So we lock spritesheets to specific specs and forbid overly high resolution in any single sheet. We go to this trouble computing memory load because on mobile interactive pages, a reload caused by an oversized image sends the bounce rate straight up. If your page goes white halfway through a scroll, check the spritesheet's source dimensions first — don't let one image eat half a phone's RAM. That isn't the browser's problem; it's ours as developers.

Loading too slowly? Three-tier loading and binary approach

Fixing the memory crash isn't enough. If users still have to wait on a long loading bar while scrolling, retention stays dismal. We designed a "three-tier load" that delivers resources in batches:

• Base tier (~2MB): the spritesheet as the skeleton, loaded first so the animation responds to scrolling the moment the page opens.
• Middle tier (~5MB): full-resolution frames loaded next, progressively replacing the blurry base preview.
• Top tier: loop handling for hold sections — the loop assets for a segment load dynamically only when the user scrolls there.

To stop users waiting idle, the download order uses "binary approach": not frame one through to the last, but jumping — 0 → 149 → 75 → 37 → 112, and so on. The payoff: once about 20% of the files have arrived, the user can already scroll through the whole visual line of the piece; a few missing frames in between don't break the rhythm.

This isn't just making the page "look faster"; it keeps the interaction alive on high-latency mobile networks. The scroll effect you see in the AI demos sits on exactly this tiered strategy — no single request for a huge file, lower server load, less waiting anxiety.

Automating frame extraction with Python

To guarantee frame quality and smoothness we wrote four Python tools to replace manual screenshots. The first is frame extraction, extract_frames.py: for a 577-frame video we recommend a 150-frame sample, and the tool distributes them precisely with round(i × 576 / 149) — better than a plain fps filter at keeping the first and last frames exact.

Add --weighted and passages with big visual change get more frames while static scenes get fewer, saving bandwidth. The full command:

python tools/extract_frames.py source.mp4 frames/ --count 150 --scale 1920:1080 --quality 72

With frames in hand, build_spritesheet.py merges them into a sheet; we set 9 columns (--cols 9) so the browser handles image memory efficiently at render time:

python tools/build_spritesheet.py frames/ spritesheet.webp --cols 9

If you're unsure about quality, run contact_sheet.py first for a preview, using --mark to flag key frames like 1, 75 and 150 and confirm the sampled frames join up as expected. These scripts removed the frame-by-frame checking we used to do, and ensure every asset entering the pipeline is already standardised.

AI footage that won't loop? Ping-pong mode and the JSON coordinate table

AI-generated motion assets have a hard flaw: the first and last frames don't join smoothly, and a straight loop visibly jumps. In testing we found that when the first-to-last difference falls between 2.3× and 7.7×, forcing a loop looks terrible. Don't obsess over making seamless assets — switch to ping-pong mode, playing the sequence forward then backward, which physically avoids the seam.

We handle this with make_loop.py, with the loop set to 24 frames to stay fluid:

python tools/make_loop.py loop-asset.mp4 hold_mid/ --count 24

With the loop sorted, mobile display is another hole. To show landscape assets precisely on portrait screens we dropped hard-coding for a JSON "coordinate timetable": for each frame it records the frame number, the horizontal offset cx (as a fraction of frame width) and the fit mode. The front end uses it while scrolling to shift and scale the background dynamically so the key image always lands centre-screen and never gets cropped by a different screen size. If your assets show blank space on phones, first check that fit in the JSON is set to cover and that cx has been calibrated to the frame's centre offset.

The two most obscure holes: sticky failing and database string limits

The first is position: sticky failing without warning. When you set overflow-x: hidden at the <html> or <body> level, the browser is forced to create a new scrolling container, which breaks sticky's positioning chain outright. The fix is simple: use body { overflow-x: clip; } instead — it cuts off overflow without creating an extra scroll context.

The second is in the back end. We wanted to store 150 frame URLs in the database and hit a text-column length limit — 150 URLs come to roughly 20KB as a string, and forcing it in gets truncated. Our final answer was "chunked storage":

• Split the 150 URLs into groups of 25, each stored in its own field.
• Reassemble on the front end and cache the result in localStorage.
• Cache for one week, avoiding frequent database reads and cutting request load at page load.

If you hit overwrite conflicts during back-end integration, add a "sequence identifier" to the data structure. It's the key marker we added while building our apps to separate asset projects, so multiple pages sharing one uploader don't scramble each other's data through ID collisions.

Quality vs performance: WebP compression and weight settings

Chasing extreme file slimming easily falls into the trap of diminishing returns. Measured: switching wholesale from JPEG to WebP saves about 53% — the most effective single move. Pushing further disappoints — dropping quality from q75 to q55 makes the picture visibly rough while saving only about 20% more. Terrible return; unless bandwidth is severely constrained, stay at q72 to q75, the sweet spot between look and size.

Another interesting number: AI-generated assets compress better than live footage. AI frames tend to be smoother in detail, coming out about 55% smaller than camera footage. But when tuning weights, never copy desktop parameters straight to mobile — mobile samples fewer frames (150 on desktop, 48 recommended on mobile), and if the weight ratio doesn't follow, scrolling stutters badly. In practice, keep mobile weights at one-third of desktop; that isn't a hunch, it keeps visual change and scroll progress in step during fast scrolling on a small screen.

Implementation guide and open-source resources

This solution is packaged as scroll-frame-sequence — 535 lines, zero dependencies, now officially open source. If you're fighting stuttery video scrolling on mobile or browser autoplay restrictions, put it straight on your list.

• Source: the GitHub repository, including the four Python asset scripts.
• Package: install from npm and import it straight into your front end.

Before you start, visit the live demo to feel the actual scroll smoothness; it fully demonstrates three-tier loading and binary approach. If you don't need database management, just generate manifest.json and host the spritesheet and frames on static storage, and you can build complex scrollytelling with this library. For edge cases like position: sticky failing or database string limits, see the bug-fixing notes in the docs — don't re-fall into the same hole in production.

Author：林政賢（Director · Gen AI creator & engineer · Founder of TangYi Studio）