Promptable Speech Recognition Moves Transcript Design Upstream

A transcript is not simply a sequence of recognized words. A meeting archive may need speaker labels and timestamps; a clinical note may require conventional spellings and normalized numbers; a multilingual support system may need translation without losing names or product terms. Two outputs can describe the same recording accurately while serving very different downstream jobs.

NVIDIA’s Nemotron 3.5 Transcribe, announced October 28, 2026, makes that output contract promptable. The open-weight checkpoint covers transcription in 40 languages, translation between English and 31 languages, and speaker attribution in 13 languages. It accepts recordings up to an hour and can label as many as eight speakers. The accompanying Granary v2 release exposes multilingual manifests and a reusable curation pipeline intended to support adaptation. Those capabilities are broad, but deploying them responsibly starts with a narrower question: what exact written artifact should an application produce?

Define the transcript contract first

Before choosing a prompt or training recipe, write a small output specification. It should state the language and script, whether disfluencies remain, how numbers and units are rendered, whether speakers and timestamps are required, and what should happen when speech is unclear. If translation is involved, specify whether names and specialist terminology should remain unchanged. For audio questions, require answers to be grounded in the recording and permit an explicit unknown result.

Consider a recorded medical consultation. A verbatim accessibility transcript might preserve repetitions and spoken number forms. A structured note might need “HbA1c” rather than a phonetic rendering and “5 mg” rather than “five milligrams.” Neither form is universally correct. The intended consumer determines correctness, and a single aggregate word-error score cannot express every part of that requirement.

Prompting is a sensible first intervention because it is cheap to revise and does not create a new model artifact. Context terms can help with unfamiliar names, but they should be framed as candidates rather than facts: the audio must still support their use. An overlong vocabulary list can encourage plausible substitutions, so test both expected terms and near-miss distractors.

Evaluate each capability separately

A unified checkpoint simplifies a pipeline, but it does not collapse distinct failure modes. Transcription can choose the wrong word. Diarization can assign correct words to the wrong person. Translation can preserve meaning while damaging a name. Audio question answering can produce a fluent claim that the recording never establishes. Each function needs its own acceptance criteria.

Build an evaluation set from the conditions the product will actually encounter: languages, accents, microphones, background noise, overlap, call length, and speaker count. Keep slices visible rather than reporting only one average. For speaker attribution, include both turn-taking and overlapping speech. For domain vocabulary, track entity accuracy alongside general word error. For questions, verify both factual support and appropriate refusal when the answer is absent.

The release reports strong public benchmark results, including 5.33% average word error rate on the English track of the Hugging Face Open ASR Leaderboard. Such results are useful for selecting candidates, not for approving a deployment. Benchmark composition, scoring rules, and audio conditions may differ from the intended workload. The model is also described as designed for recorded audio rather than live streaming, an important boundary for real-time products.

Treat adaptation as a data decision

Granary v2’s public manifest covers 463,062 hours across 48 languages and 16 source corpora. It includes three transcript forms: text-normalized verbatim, inverse-text-normalized with disfluencies, and inverse-text-normalized without them. That design illustrates a general rule: training examples must make the desired written convention explicit. A mixed dataset with inconsistent punctuation, number formatting, or treatment of filler words teaches ambiguity rather than flexibility.

The pipeline also attaches provenance and quality information and derives items such as translations, contextual terms, and speech-grounded question-answer pairs. Teams using the manifests should still inspect the license and provenance of each underlying corpus, obtain source audio according to its terms, and decide whether automated labels are suitable for their purpose. Dataset scale is not a substitute for permission, relevance, or label review.

Fine-tuning becomes reasonable when a stable prompt still misses important patterns. The accompanying medical case study reports a held-out speaker WER reduction from 22.44% with the best prompt to 12.17% after adaptation, while LibriSpeech test-clean WER rose from 1.54% to 1.69%. That tradeoff is instructive: specialization can improve the target domain while slightly weakening general performance. Preserve a general-speech regression set and evaluate the exact entities that matter rather than optimizing only a headline average.

Use a staged adoption path

Start with a frozen development set and one explicit transcript contract. Establish a no-context baseline, then test a short task prompt, then add carefully bounded terminology hints. Record the model revision, decoding settings, prompt, and scorer for every comparison. Only after prompt changes plateau should the team prepare representative labeled audio for adaptation.

Keep the final test set isolated from prompt design and checkpoint selection. Review errors by category: omissions, substitutions, speaker swaps, formatting violations, unsupported answers, and regressions on ordinary speech. Human review remains especially important for names, decisions, medical terms, and any transcript that drives an automated action.

Nemotron 3.5 Transcribe reduces the number of components needed to explore several speech tasks, while Granary v2 makes transcript construction and data curation more inspectable. The main opportunity is not merely to replace a conventional ASR endpoint. It is to turn transcript design into an explicit, testable interface—and to adapt only when evidence shows that prompting cannot meet it.