Quantize the lifter, not the detector

OneRep's Form Coach takes a video of you lifting and turns it into numbers (joint angles, tempo, range of motion, how lopsided you are), and it does all of the looking on your own phone, because your video is yours and we are apparently the last people left who think that sentence should end there.

Which means two neural networks have to run in a WebView, on a phone, over a fifteen-second clip, fast enough that you don't wander off to make tea. Everything below is the residue of that constraint. Pour yourself something. I did.

The pipeline

Two stages, and you need to hold on to exactly one fact about them:

1. YOLO11n-pose reads each sampled frame and returns 17 2D keypoints in COCO order. It runs once per frame.
2. MotionBERT-lite takes the whole sequence of keypoints and lifts it into 3D, using the frames around a frame to resolve the depth that a single frame cannot. It runs once per clip.

We sample at 12 fps. Fifteen seconds is about 180 detector calls and one lifter call. One hundred and eighty to one. That ratio is the entire article; everything after this is me making you feel it.

The obvious optimisation, applied with the obvious confidence

Both models went to ONNX, and both looked like candidates for int8 quantization. The MotionBERT checkpoint is 64 MB of fp32, which is an obscene thing to push down a phone connection, so quantizing it seemed clearly right. Quantizing the detector seemed clearly right too, for the size and for the folk wisdom: everyone knows lower precision means faster inference. Everyone knows a lot of things at the start of a project.

One of the two was correct.

The detector got ten times slower

Single-threaded on wasm, the int8 detector benchmarked at 206 ms per frame against 21 ms for fp32.

Read that again. Not slower by a margin you'd argue about in review. Ten times.

The cause is so mundane it's almost insulting: onnxruntime-web has no fast int8 convolution kernel on wasm. When it meets a quantized convolution it shrugs, dequantizes the weights, and runs the whole thing in floating point anyway: full fp32 cost, plus a dequantization pass, on every inference. You get the smaller download and one tenth of the speed. A diet where you keep the weight and lose the muscle.

Across 180 frames that is four seconds versus thirty-seven. Nobody waits thirty-seven seconds for a squat verdict. I barely waited four.

So the detector ships fp32, at 11 MB, and the optimisation went in the bin where it belonged.

The lifter had to be quantized anyway, and not for the reason you'd guess

Cloudflare Pages, which serves our static assets, has a hard 25 MiB per-file cap. The fp32 MotionBERT graph is 64 MB. You cannot negotiate with a CDN. You cannot even get it on the phone. Either the model gets smaller or the architecture changes, and one of those is a weekend and the other is a quarter.

int8 takes it to 16.6 MB. Accuracy cost against the fp32 model: small, and measured. Speed cost: about 11%, on an operation that runs once per clip, in under a second, while nobody is looking.

Eleven percent slower, once, invisibly, in exchange for the model being deliverable at all. Some trades you agonise over. This was not one of them.

The rule, since you'll want one

The two decisions look contradictory and are the same decision, applied to two different call frequencies:

Quantize what you download. Do not quantize what you execute in a loop, not until you have watched your actual runtime execute actual int8 and timed it.

That last clause is the load-bearing one. This is a fact about onnxruntime-web on wasm, not about quantization. Our CoreML export of the same detector has no such problem; the Neural Engine has real int8 kernels. (We ship that one fp16 regardless; one accuracy story across backends, rather than a discrepancy that only manifests on iOS and only at 2 AM.)

What it cost to learn, itemised

Benchmark the runtime, not the format. "int8 is faster" is a claim about hardware with int8 execution units. In a wasm sandbox it can be a claim about nothing at all. We found this with bench_onnx.py, a deliberately boring single-threaded latency harness. And, since we're being honest with each other at this hour, we found it after shipping the wrong thing first.

A file-size cap is architecture, whatever it thinks it is. The 25 MiB limit appears in no model card and no paper, and it decided which precision we ship, which is to say it decided a real accuracy trade-off in a real product. Infrastructure limits reach much further up the stack than their documentation admits. They always do.

Related

Rep counting on top of this pipeline had its own inverted-intuition problem, written up in counting reps without knowing the exercise. Same lesson, different organ.

Form Coach ships in OneRep. Every account gets a monthly AI allowance (a filmed set costs two of it, so five analyses a month free), and the €4.99 Coach subscription raises the allowance rather than ransoming the feature. The tracker underneath, workouts, food and progress, is free and unlimited, a business model I'm assured is unwise.

Compare: OneRep vs Hevy