From a video file to a streamed structure
Every stage is a real script in the repository, and every intermediate artifact is on disk and inspectable. Nothing in this chain is a black box.
- Source mediaA single mp4. The baseline the rest is derived from.
- Frames, masks, skeletonPer-frame extraction of regions and motion structure.
- Temporal smoothingSkeleton smoothed across frames so motion does not jitter between adjacent states.
- S0 coarse texturesLow-frequency structure per frame.
- S1 signed residualThe correction layer, stored because it cannot be regenerated.
- S2 coefficientsFrequency detail, kept as coefficients rather than pixels.
- Adaptive semantic meshL0/L1/L2 meshes at 20, 42 and 110 vertices, selected by quality and distortion.
- Registry-driven bundleA deployable directory the runtime and the tests both consume.
P0–P4: degrade to something inspectable
Backends initialize lazily and fall back rather than fail. A backend that cannot start hides its own button instead of leaving a dead control, and the runtime drops to the next path down.
Structured GPU stream
Streamed textures, adaptive mesh, S0+S1+S2 on the GPU.
Procedural WebGL
Shader reconstruction without the streamed asset package.
Structured CPU / Canvas
Same structure, drawn on a 2D canvas. Inspectable.
Sprite atlas seek
Pre-rendered states in an atlas.
Linear video seek
The baseline the whole project is arguing with.
How it is verified
11 test suites cover the state store, dirty graph, spectral layout determinism, the structured pipeline, temporal smoothing, deformation and residual codec, the GPU mesh path, streaming and adaptive LOD.
Visual regression renders the golden references fresh and compares them within a tolerance rather than by hash. Byte-exact comparison cannot work: the renderer uses the platform rasterizer, so a different build draws the same picture with different antialiasing. Measured legitimate difference is 0.55% of pixels differing by more than 24, with identical mean colour. The tolerance sits about four times above that, and was checked against real regressions — a six-pixel shift and a twelve-degree hue change both fail it.