Precision, Runtime and Platform in the Deployment of an Audio Deepfake Detector: A Measured Cross-Target Study
Audio deepfake detectors are increasingly proposed for use outside the datacentre, but published work reports detection accuracy far more often than the cost of running the model.
This study measures one 158.9M-parameter dual-stream detector across seven inference configurations spanning three device classes: an NVIDIA L4 GPU, a server CPU, and a 2021 Android handset, with three numerical precisions and four runtimes.
Every configuration scored the identical 4,000-clip, 11.27-hour out-of-distribution evaluation set, so each comparison is clip-paired and carries an explicit statement of what it holds fixed.
Across the seven configurations sustained capacity spans 312x while ROC-AUC spans 0.00040: for this model, choosing a deployment target is almost entirely a systems decision rather than a detection-quality one.
Out-of-distribution by construction: every configuration scored the identical clip set, in the same order, so comparisons are clip-paired.
One detector, one graph, three device classes, each measured on the figure that matters for where it runs.
FP32 PyTorch on the L4 is the reference point at 43.9 ms median per clip. Casting the weights to FP16, with the same code path, thread count and warm-up policy, drops the median to 14.4 ms, a 3.09x per-clip speedup, while the artifact halves from 606.3 MiB to 303.2 MiB. Only 2 of 4,000 labels change.
At fixed runtime, ONNX Runtime on both sides, dynamic INT8 cuts the median from 498.9 ms to 420.9 ms, about 1.18x, while compressing the artifact from 522.6 MiB to 177.7 MiB, roughly 2.94x smaller. The aggregate speedup is smaller than the arithmetic one because a fixed ~91 ms per-call overhead does not scale with audio length, so short clips benefit least.
The byte-identical INT8 graph was moved from the server CPU to the handset, so the platform is the only thing that changes. Detection behaviour holds: ROC-AUC differs by 1.0 x 10^-5, 3,993 of 4,000 labels are unchanged, and the exact McNemar test on the discordant pairs gives p = 1.0. For this model and this graph, the handset delivers the same detection quality as the server.
Judged against the actual duration of each clip, the handset stays ahead of the audio for 3,986 of 4,000 clips, 99.65%, at a median real-time factor of 0.634, with no accelerator delegate engaged: CPU-only, four of eight cores.
Across the seven configurations, sustained capacity spans 312x while ROC-AUC spans 0.00040. The device moves cost by factors of nine to fourteen at each step; the largest precision gain available within a device is 3.09x. What limits deployed accuracy is neither the device nor the numerics but the decision threshold.
Full text, 25 pages, rendered here on the page. No download.
Evaluate Detectif.ai on your own channels, codecs, languages and generators.