2020 thesis project · Live demo

AiRoll.

An AI movie editor. Drop in a few clips and a song: AiRoll finds the beat, reads each clip's shot type with a DINOv3 classifier and cuts a montage to the music. It all runs here in your browser, so your footage never leaves the page.

GitHub Classifier explorer Model Dataset rucinski.ai
7 shot sizes the model tells apart
Shot sizes
7
wide to detail
Classifier
DINOv3-S
int8 ONNX, 22 MB
Training set
53K
labelled film stills
Accuracy
70.4%
859 gold frames
Beat tracker
Custom
built in JavaScript
Runs in
Browser
WASM, no server
01 /

Cut a montage

Movie clips
Music track
Loading…
in the montagerest of windowbeatsDrag across the wave to pick the music

Short clips (a few seconds each) and a song with a clear beat work best.

Beat-cut montage

02 /

How it works

1 · Beat

Find the beats

Web Audio decodes the song. The JavaScript tracker implemented here builds an onset-strength envelope from the mel spectrogram, estimates tempo from the envelope's autocorrelation, and uses dynamic programming to place beats on strong onsets at a steady tempo. Beats are tracked across the whole song, then aligned to the chosen music start/end window.

2 · Shot

Read the framing

One middle frame per clip is resized to 256 px and centre-cropped to 224. DINOv3-small, fine-tuned on film stills, classifies 7 shot sizes. The 22 MB int8 ONNX model runs in ONNX Runtime Web via WebAssembly and downloads once, with your first own clip. Sample clips use precomputed labels.

3 · Cut

Render the edit

Each beat marks a cut. Clips take turns, each cut from a fresh stretch of its clip, rendered at 360p with the shot size label burned into the top-right corner. WebCodecs encodes 24 fps H.264 video and AAC or Opus audio from the chosen music window. Mediabunny writes the MP4. Nothing is uploaded.

4 · Edit

Refine the cut

The timeline keeps every cut on its beat. Drag cuts to reorder them, slip a cut to choose a different moment of its clip, or delete it and let the rest close the gap. Hover to skim, undo any step, then render again. Each clip card marks the stretches the edit uses.