My clipper can't predict virality. I tested it, then shipped it

Gemma 4, a 26-billion-parameter open model, scored 0.56. A coin flip scores 0.50.

The question: given a long talk, can a model tell which moments viewers will replay? I committed the test to the repo before a single clip was judged, then ran it on 19 Creative Commons videos, 3,591 candidate clips in all. None of my four hypotheses was supported.

I shipped the clipper anyway. This post is about why, and what it does instead of predicting virality.

What you’ll find here:

• The pre-registered eval, and how its negative result shaped the product.
• “Faithful cuts”, a check for clips that change what the speaker meant.
• What broke on Nigerian-accented English, and the fix.
• What leaves your device, and how to rerun my test.

Selects is a clipper that asks before it cuts

Selects takes a long video and turns the best moments into captioned 9:16 clips. In film editing, “selects” are the best takes an editor pulls from raw footage.

The laptop version uses whisper.cpp for the transcript, Gemma through Ollama to judge, and ffmpeg to cut. It adds an overnight queue, a Nigerian-accent speech model, face and region blur, and a caption-only mode. Selects Web runs the same pipeline in a browser tab.

Three rules run through both:

1. Gemma explains every pick. Each clip shows four scores (hook, stands alone, payoff, clarity) and a one-line reason you can disagree with.
2. Nothing renders until you approve it. The review screen, which I call the Reel, shows one clip at a time beside a clapperboard slate. Press A or swipe right to approve, and the card is thrown onto the pile. Press S or swipe left to skip.
3. Every cut starts and ends on a sentence. Trims snap to sentence boundaries from whisper’s word timings.

I started it for people who clip long videos for money. Hosted clippers charge a subscription: Opus Clip’s Starter plan is $15 a month. Selects has no subscription and no watermark.

The test I wrote before I looked

The pitch for an AI clipper is that it finds your best moments. I wanted to know whether a model could do that from the words alone. So I needed ground truth that I didn’t choose myself.

I used YouTube’s “Most Replayed” graph. It shows where viewers rewind and rewatch, in about 100 segments per video. It is a proxy, not a definition of a good short. A lot of replays happen around visual moments, and Selects only reads the transcript. I wrote that caveat into the plan before running anything.

The plan, in eval/PLAN.md, was committed on 17 October before any judging. It fixed four primary hypotheses:

• H1. Gemma 4 ranks clips by replay intensity better than chance.
• H2. Gemma 4 beats Selects’ simple first-stage rules (the “pre-score”).
• H3. The pre-score beats chance.
• H4. Gemma 4 beats a loudness and pace baseline that reads no words.

The metric was AUC: take one clip viewers replayed more than the video’s median and one they replayed less, and ask how often the method scores the replayed one higher. A coin flip gets 0.50. Each video counts once, and the four tests are corrected for multiple comparisons (Holm’s method).

The videos were 19 CC BY talks, interviews and lectures, including Nigerian and Ghanaian channels. Four more were excluded before judging: three because whisper’s transcript was unreliable, one because the audio was dubbed. One of the three was a Kenyan podcast, so East African voices are missing from the result. Gemma 4 ran on Google’s free Gemini API: 895 calls, $0.

The result: barely better than a coin flip

Gemma’s interval includes 0.50. The pre-registered test for H1 gave p = 0.107, or 0.428 after correction. H2, H3 and H4 were not supported either. The full tables are in eval/results/RESULTS.md.

The spread across videos was wide. On a DEF CON talk, Gemma reached 0.82. On an IELTS speaking test, it scored 0.25, worse than chance.

Two exploratory numbers, labelled as such because I didn’t pre-register them. Blending Gemma’s score with the pre-score, which is how the app actually ranks clips, reached 0.57 (0.51 to 0.64). A 0.5-billion-parameter model blended the same way also reached 0.57. At the effect sizes I saw, about 52 videos would give a fair chance of detecting Gemma’s edge over chance. I had 19.

So here is the honest claim. Nothing I tested, from loudness to a 26B model, predicted replays from the words alone at a level I can tell apart from chance. That doesn’t prove it’s impossible. It means I can’t sell it.

That result changed the product. Selects doesn’t show a virality score, because I can’t back one. It shows the model’s reasons, and it leaves the decision to you.

Faithful cuts: does the clip still say what the speaker meant?

Ranking clips is one problem. Cutting them honestly is another. A clip that stops one sentence early can flip a speaker’s meaning.

So Selects checks each top pick again, with the sentence before and after. A rule looks for cues such as a turn (“But…”) or reported views (“they think…”). Gemma answers five narrow questions about what the next sentence does to the clip’s point. A short written rule turns both into a verdict: faithful, check or misleading. The note quotes the words outside the clip and suggests a fix, and nothing changes until you say so.

Here is a real one, from Sam Altman’s first “How to Start a Startup” lecture (CC BY 3.0). The clip, 3:59 to 4:26, ends:

And great execution is at least 10 times more important and 100 times harder than a good idea.

The next sentence:

But the pendulum has swung way out of whack here.

On its own, the clip sounds like advice to skip thinking about the idea. The lecture argues the opposite. Selects marked it misleading (a turn) and suggested ending at 4:28.

Another, from Jenni’s EMF 2026 talk on strength training (CC BY-SA 4.0). The clip ends:

Another thing that sometimes people say when I kind of try to advocate for weight training is that they think they’re too old to start.

The speaker’s next words are “And that’s definitely not true.” A viewer of the clip hears the opposite.

Both examples came from local Gemma 3 4B runs on my laptop, documented in notes/faithful-examples.md. In the eval, with Gemma 4, the check flagged 10 of 95 top picks (11%, 95% CI 6% to 18%) as “check” or “misleading”: 3 quotes, 3 turns, 2 missing qualifiers, 2 references. What I haven’t measured is precision: how many of those flags a careful reader would agree with. The 20-example review sheet is in the repo, waiting for marks.

Accents broke the language check first

Whisper’s built-in language detector listens to the first 30 seconds and guesses. On my Nigerian-accented English test audio, it said Yoruba, with p = 0.90. On a British conference speaker, it said Welsh (p = 0.63). If Selects had trusted it, it would have transcribed English as Yoruba.

The fix was to stop trusting one signal. Selects now asks whisper to decode short samples in each likely language, then has two small text models read what whisper wrote: lid-lite-608 (608 languages, 37 MB) and lid-lite-5 (English, Hausa, Igbo, Nigerian Pidgin and Yoruba, 1.1 MB). Told to write Yoruba, whisper wrote English, and both text models read it as English (0.994 and 0.999). On 10 test recordings, the ensemble got 9 right and asked a question on the 10th (a Hausa set) instead of guessing.

The transcript itself was the next problem. With the default English model, the run stopped before judging: its transcript quality score was 0.638, under the 0.65 line. When the language check finds signs of Nigerian English, Selects now switches to a community whisper model fine-tuned on Nigerian-accented English. The same audio scored 0.915. That score is whisper’s own confidence, not measured accuracy, and nobody has rated those transcripts yet. Check the captions before you post.

It runs on a 2017 laptop, or in a browser tab

I built Selects on a 2017 13-inch MacBook Pro: an Intel i5 with 2 cores, 16 GB of RAM and no GPU. With Gemma 3 4B judging locally, a 6-minute talk became 3 captioned, credited clips in 21.6 minutes, at about 73 seconds per judge call.

For long videos, transcription runs in 10-minute parts and every stage is saved, so a killed job resumes where it stopped. A 1-hour source became 30 previews and 5 finished, credited clips in 36.6 minutes, with peak memory of 904 MB. One caveat: that run judged with Gemma 4 through the Gemini API. With local Gemma, long videos are a job for the overnight queue.

Selects Web moves everything into the tab: whisper through transformers.js, Gemma 4 E2B through Google’s MediaPipe runtime on WebGPU (or Gemma 3 1B on the CPU), and the render through WebCodecs. On the same laptop, 2 minutes of video took 11.2 minutes to analyse, and a 41-second clip rendered in 57 seconds. It works best for videos up to about 10 to 15 minutes.

What leaves your device

Selects Web has two modes, and the difference is the whole privacy model.

Smart mode uses your own free Google AI Studio key, kept in the tab, or a small Selects proxy that rate-limits visitors and doesn’t log the text. Google’s free tier may use that text to improve its products, and the app says so before you switch. If Google is slow, the clip is judged on your device instead.

The laptop version uploads nothing.

Limitations

• No virality prediction, and a small eval. 19 videos, not a random sample of talks, against a proxy (Most Replayed) that isn’t “a good short”.
• Faithful-cut precision is unmeasured. The flag rate is known; how often the flags are right is not.
• Accented English is improved, not solved. The fine-tune’s transcripts are unrated, and whisper’s small models transcribe Yoruba and Hausa poorly. Igbo has no whisper model at all.
• Audio and text only. The model never sees the video, so it can’t tell when a moment depends on a slide or a demo.
• Slide talks usually render as “fit” (the full frame over a blurred copy). Face blur needs macOS.
• Selects Web is English-only, best under 15 minutes, and the first run is a large download.
• Tested on macOS 13 on one machine. Linux hasn’t been tested.

Try it, and check my work

The repo is MIT licensed: github.com/IkpemosiArnold/selects. The README covers the laptop install, and selects doctor checks every piece. To try it without installing, open selects.ikpemosi.work in Chrome or Edge, keep Private mode on, and give it a short video you have the rights to.

The plan, the cached model answers and the report generator are all in the repo:

git clone https://github.com/IkpemosiArnold/selects
cd selects && npm install
node eval/run-eval-v2.mjs --report   # rebuilds RESULTS.md from the cached answers

There is also a blind test: two clips from the same video, one picked by Selects, and the question “Which would you rather post?”

node eval/prefer.mjs           # this computer: http://127.0.0.1:4319
node eval/prefer.mjs --share   # phones on the same Wi-Fi, with a private link

It has 30 pairs and takes about 10 minutes; without the source videos, you rate from the transcripts. I’ll publish the result, whichever way it goes.

Credits and licences

• Selects: MIT, © 2026 Arnold Ikpemosi Adeyemi.
• Models: whisper (MIT) via whisper.cpp; Gemma 3 (Gemma Terms of Use); Gemma 4 E2B and 26B (Apache-2.0); rishabbahal/whisper-small-nigerian-accent (Apache-2.0); olaverse lid-lite-608 and lid-lite-5 (Apache-2.0); Edge-Punct-Casing via sherpa-onnx (Apache-2.0). Full list with sizes: notes/MODELS.md.
• Quoted: “Lecture 1 - How to Start a Startup (Sam Altman, Dustin Moskovitz)”, YC Root Access, YouTube, CC BY 3.0.
• Quoted: “How I became (nearly) as strong as the average untrained man”, Jenni, EMF 2026, media.ccc.de, CC BY-SA 4.0.
• Eval videos: 19 CC BY videos, listed with links in RESULTS.md.

Why I’d rather ship a reason than a score

Before I wrote software, I spent five years on climate programmes, where nothing counted until it was written down and someone else could check it. Pre-registering this test was the same habit. It cost me the headline I wanted, and it gave me a better product.

If you clip for a living, run Selects on a video you’d normally cut by hand, and open an issue for whatever it gets wrong.