We Tested VibeVoice-ASR-BitNet for Local Dictation. Here Are the Results
Local dictation should not stop working because a GPU driver, cloud provider, or network connection is having a bad day. DictatorFlow now understands Microsoft's VibeVoice-ASR-BitNet model directory as a first-class local transcription backend.
Why this model is interesting
Microsoft's model card describes a 1.58 GB quantized model designed for commodity x86 AVX2 and ARM NEON CPUs. Its published AMD EPYC test reaches a real-time factor below 1 with three threads and improves as more threads are added. That makes it a practical offline option instead of an emergency fallback that is too slow to use.
Accuracy still matters more than a small benchmark number. Microsoft reports 8.25% WER on MLC English, 21.36% on AMI close-mic meetings, and 2.41% on LibriSpeech clean. Those figures come from the model authors, not our production traffic, so DictatorFlow's evaluation harness keeps reporting per-engine latency and WER on the same recordings before routing policy changes.
On DictatorFlow's small nine-clip English corpus, deterministic greedy decoding averaged 60.45% WER and a 1.53 real-time factor with eight threads. The runtime loaded in 2.89 seconds on a dual-socket Intel Xeon E5-2697 v4. Some natural clips were useful, including 11.4% WER on the longest sample, but technical phrases, numbers, pangrams, and synthetic speech failed badly enough to reject a production rollout.
This is a small product-specific corpus on older hardware, not a universal verdict on Microsoft's published benchmarks. It is enough to answer our deployment question: we kept the existing local and cloud defaults and did not redeploy production traffic to BitNet.
- > A model directory containing both quantized GGUF files is detected automatically.
- > The official VibeASR.cpp streaming server stays warm, so the model pair is loaded once rather than once per recording.
- > Audio is passed locally as a temporary PCM WAV and removed as soon as transcription finishes.
- > A crashed local server is restarted once automatically before the request fails.
CPU, NVIDIA, and AMD
VibeASR.cpp's current BitNet path is CPU-only by design. DictatorFlow therefore keeps the existing Parakeet ONNX backend for GPU users: CUDA remains the NVIDIA path, while ROCm-enabled ONNX Runtime builds can select the AMD execution provider. If either GPU provider is unavailable, the ONNX model falls back to CPU. VibeVoice itself uses its optimized CPU kernels on machines with NVIDIA or AMD graphics until the upstream runtime adds a supported GPU implementation.
That separation is deliberate. "AMD supported" should mean a tested execution provider, not a CUDA flag renamed for marketing. AMD CPUs are already a strong target for the BitNet engine; AMD GPU acceleration is available for the ONNX backend when DictatorFlow is built against ONNX Runtime ROCm.
Install the offline engine
curl -fsSL https://dictatorflowstatic.dictatorflow.com/install-vibevoice-asr.sh | bash
# Then test a 16 kHz f32le recording:
dictatorflow --test-local recording.pcmThe installer downloads only the two ready-to-run quantized files, builds Microsoft's pinned runtime, and verifies that both models load. It deliberately does not select the engine by default. Run e2e/vibeasr_eval.py on representative recordings first, then rerun the installer with DICTATORFLOW_VIBEASR_UPDATE_CONFIG=1 only if the result is acceptable. Set DICTATORFLOW_VIBEASR_THREADS to tune CPU use.
DictatorFlow can now evaluate VibeVoice through a warm local process in both the desktop client and API server. CUDA, ROCm, Parakeet CPU fallback, and cloud providers remain unchanged. BitNet stays opt-in until a broader corpus and newer CPU results justify changing that decision.