NVIDIA NemotronLabs VoiceChat 11B: The Full-Duplex Voice Model NVIDIA Shipped Without Announcing It

Two model cards sit inside the same Hugging Face repository, and they disagree with each other. The one that renders on the page calls the model NVIDIA NemotronLabs VoiceChat 11B, gives its release date as August 3, 2026, and lists 11B parameters. The second, a file named overview.md that nobody sees unless they open the Files tab, calls the same model 12B, dates the release July 16, carries a benchmark section the public card leaves out, and still has the word TODO sitting where a results description belongs. Neither card was accompanied by a launch post, a press note, a tech report, or a tweet from NVIDIA.

What is in there is not a placeholder, though. It is a 44 GB checkpoint that listens and speaks at the same time: a single stack that takes microphone audio in and pushes synthesized speech out, without the usual relay of speech recognition to a language model to text-to-speech. It is built on the Nemotron Nano 9B v2 backbone, it can call tools mid-sentence while it keeps talking, and NVIDIA claims it is the first open full-duplex model that can do that.

Everything below is read directly off that repository — the two cards, config.json, and the tensor index inside the weights file, which we parsed ourselves to settle the 11B-versus-12B question. Every performance number NVIDIA publishes for this model is NVIDIA's own, measured by NVIDIA, and unreproduced. Exactly one independent measurement of this lineage exists in public, from Artificial Analysis, and it is filed under a different name and a different parameter count — which turns out to be the most interesting thing about the release.

What is actually in the repository

The repo was created on July 29, 2026 and last touched on August 4. It contains seventeen files: the two competing model cards, a license, a config.json, one 44.4 GB model.safetensors, an architecture diagram, an RNN-T tokenizer directory, four short policy notes on bias, safety, privacy and explainability, and three .wav files — a turn-taking demo, a barge-in demo, and a tool-calling demo — embedded as audio players at the top of the card so you hear the model before you read about it.

Several things are absent, and they matter more than the file list.

• There is no paper for this model. The card cites five: VoiceBench, Full-Duplex-Bench 1.0, Full-Duplex-Bench v3, SALM-Duplex, Audio Flamingo 3, plus NVIDIA's own PersonaPlex preprint. None of them is a NemotronLabs VoiceChat tech report. The architecture is described in about three paragraphs and a diagram, and that is the whole public account of how it was built.

• There is no way to call it. The Hugging Face page states plainly that the model "isn't deployed by any Inference Provider." There is no hosted endpoint, from NVIDIA or anyone else. The single open community thread on the repo is a user asking NVIDIA to add a handler.py so the checkpoint can be deployed to Hugging Face Inference Endpoints — someone who tried to run it the easy way and found there isn't one.

• There is no pipeline_tag and no library_name. That is why the page has no inference widget and no task label, and it is a symptom of the deeper issue: config.json is not a Transformers config. It is a 32 KB NeMo training config, complete with an AdamW block, a learning-rate schedule, dataloader bucket bins and a validation split. You cannot point AutoModel.from_pretrained at this. You clone a specific branch of NVIDIA's Speech repository — nemotron-labs-voicechat — build a conda environment pinned to Python 3.12, PyTorch 2.10 and Transformers 4.56, compile causal-conv1d and mamba-ssm without build isolation, and run their script.

The download counter reflects all of that. 107 likes against 80 downloads in the model's first month is the signature of a release people bookmarked and did not run.

We counted the parameters, and 11B wins

A safetensors file carries a JSON header listing every tensor, its shape and its dtype, so the parameter count is not a matter of opinion. We pulled the header and added it up: 11,095 million parameters across 1,626 float32 tensors, occupying 44.38 GB. So the rendered card is right and overview.md is stale — this is an 11B model, and the 12B figure is a leftover.

Hugging Face's model tree explains where a 12B could have crept in. The lineage it draws runs Nemotron Nano 12B v2 Base, then Nemotron Nano 12B v2, then Nemotron Nano 9B v2, and only then this model. NVIDIA's own text backbone family contains both a 12B and a 9B, the 9B being the pruned descendant of the 12B, and VoiceChat is built on the 9B. A card drafted earlier in that chain would naturally have carried the larger number.

The header also splits cleanly along the two halves of the architecture:

• Understanding side — 10.098B. Of that, 7.714B is the Nemotron Nano v2 transformer stack, 0.609B is the speech encoder, and three separate 131,072 × 4,480 matrices take 0.587B each.

• Speaking side — 0.997B. A 28-layer, 1,152-wide text-to-speech backbone at 0.595B, a 0.159B mixture-of-Gaussians output head, and a neural audio codec whose encoder and decoder are 0.092B apiece.

Two practical consequences fall out of the dtype. The first is that the 44 GB download is not a big model, it is an ordinary model shipped in float32; cast it to bfloat16 and the weights are roughly 22 GB. The second is that NVIDIA's stated floor of an 80 GB GPU is a consequence of that choice rather than of the model's size, and anyone with a 48 GB card who is willing to convert the checkpoint themselves is not obviously locked out — though nobody has published a confirmed run at that footprint, so treat it as arithmetic, not a promise.

How it listens and talks at the same time

"Full duplex" is the whole point of the release, and the config shows how it is bought. Three design choices do the work.

• The speech encoder cannot look ahead. It is a 24-layer, 8-head Conformer whose attention context is set to [70, 0] — seventy frames of left context and zero frames of right context. A conventional recognizer peeks at audio after the current moment to disambiguate what it just heard; this one is forbidden from doing so, which costs accuracy and buys the ability to emit a decision the instant a frame arrives.

• Everything is quantized to 80 ms. The frame length in the config is 0.08 seconds, matching the 80 ms chunk size NVIDIA has described elsewhere for this line of work. The model is deciding, every 80 ms, whether to keep listening or to start speaking. That cadence is what makes barge-in feel immediate rather than polite.

• Tool calls come out of their own mouth. Remember those three identically-shaped 131,072-vocab matrices. One is the input embedding, one is the ordinary language-model head — and the third is named function_head. The "separate output channel for tool-calling scripts" in the card's prose is a literal third output projection, 587 million parameters wide, running in parallel with speech generation. That is the architectural reason the model can dispatch a tool call without stopping the conversation, and it is a real design commitment rather than a prompt convention.

Downstream, the text-to-speech decoder predicts codes for a residual-vector-quantized codec with 31 quantizers and downsampling rates of 7, 7 and 9 — a 441× reduction that puts the audio token rate at 50 frames per second, out at 22.05 kHz. Input is 16 kHz. There is also an RNN-T decoder and joint network in the checkpoint, which is why the model's declared outputs include a transcript of the user alongside its own text and audio: the transcription is a real recognition head, not a by-product. The default voice is called Aria, and a three-second reference clip conditions the speaker.

One more detail from the config is worth flagging because it corroborates a stated limitation: the training dataloader caps utterances at 142.39 seconds. The card's warning that the model holds no more than a two-minute audio context is visible in the data pipeline that produced it.

The scores, and who actually measured them

NVIDIA's headline claims are latency and rank. On Full-Duplex-Bench 1.0 it reports 448 ms to take a smooth turn and 480 ms to yield when interrupted, with a takeover rate of 0.82 on smooth turn-taking and 1.0 on user interruption, and a GPT-4o judge score of 4.33 on interruption handling. It claims #2 among open full-duplex models on VoiceBench and #2 among open models on Full-Duplex-Bench. All of those are NVIDIA's own runs.

Line them up against the outside world and two gaps open.

• On speech reasoning, the vendor number is about eight points high. The unrendered overview.md reports 37.0% on Big Bench Audio. Artificial Analysis independently measured this lineage at 29.2%. Same benchmark, same family, a gap large enough to change where the model sits in the ranking.

• On VoiceBench, the vendor number is too modest. The card claims #2 among open full-duplex models; the public VoiceBench leaderboard has this model at 58.10, which is the top of the open full-duplex bucket, ahead of Freeze-Omni at 55.20 and Moshi at 29.51. NVIDIA's own draft card reports 55.1 for itself — below the number the leaderboard now lists.

There is a smaller oddity too: NVIDIA's own comparison table rates its rivals more generously than Artificial Analysis does. The draft card puts Freeze-Omni at 33.4% and PersonaPlex 7B at 19.1% on Big Bench Audio; Artificial Analysis measures 33.9% and 12.6%. The peer ordering survives either way, which is reassuring, but it is a reminder that a vendor's harness and an independent one do not return the same numbers even for third parties.

The chart makes the honest headline unavoidable. Among open full-duplex models this is a genuine step forward — it more than doubles PersonaPlex on spoken reasoning and leaves Moshi in a different category entirely. Measured against what you can buy, it is nowhere close: Step-Audio R1.1 at 96%, Grok 4.5's Voice Agent and Gemini 2.5 Flash (Thinking) at 92%, Nova 2.0 Sonic at 87%. That is roughly a three-to-one gap on reasoning from spoken input.

The cleanest way to see what full duplex currently costs is inside NVIDIA's own catalogue. On VoiceBench, NVIDIA Nemotron 3 Nano Omni 30B A3B — a bigger model that is not full duplex — scores 89.39, against 58.10 for VoiceChat. Roughly thirty points of assistant quality is the price of the interruption handling and the sub-500 ms turn-taking, at least in this generation. If your product does not need a model that can be cut off mid-word, you are paying a lot for a feature you will not use.

Tool calling is the headline, and also the weakest part

"First open full-duplex model to support tool calling" is the claim the release rests on, and the numbers underneath it are uneven. On a spoken conversion of BFCL-v3, NVIDIA reports a 56.1% average: 58.5% simple, 62.5% multiple, 42.5% parallel, 27.5% parallel-multiple, and 89.6% on correctly declining irrelevant calls. On Full-Duplex-Bench v3 the split is sharper still.

• Tool selection — 82.5%. Picking the right function from the list, which NVIDIA fairly describes as competitive with frontier models.

• Argument accuracy — 44.2%. Filling that function's parameters correctly from what the user said.

• Pass@1 — 33%. Getting the whole call right on the first attempt.

A model that knows which tool it wants two-thirds more reliably than it can fill in the tool's arguments is a model that will confidently look up the weather for the wrong city. In a text agent you would catch that with a validation layer and a retry; in a live voice call, the retry is audible, and the card notes the model should not retry a failed call at all — it is instructed to tell the user the API has a problem.

The operational constraints around it are tight, and NVIDIA lists them without much spin: a maximum of five tools per session before quality degrades, no reliable simultaneous multi-tool calls, no way for the user to interrupt while a tool is executing, and a tendency in mixed conversation to answer from its own knowledge instead of calling the tool it should have. Long tool responses stall the agent, which is what the "on-hold message" feature exists to paper over — you pre-write a phrase the agent speaks the moment a call fires, so the silence has something in it.

One quirk that will cost somebody an afternoon: system prompts and tool responses must be ASCII only. No em dashes, no curly quotes, no degree symbols, no emoji. Tool output has to be flattened into plain speech-friendly ASCII sentences before it reaches the model, which means a JSON API response can't be passed through raw.

What it costs to run, and the license that stops you

Start with the hardware. NVIDIA's branch documentation asks for a GPU with at least 80 GB of memory, which points at an H100 or H200, and the tested configuration is an H100 with vLLM. On-demand H100 capacity runs roughly $2–3 per hour across the common GPU clouds, so a continuously-running instance is somewhere near $1,500–2,200 a month before you have written any application code, hired anyone to keep it up, or served a second concurrent conversation.

Now the comparison. Artificial Analysis normalizes hosted speech-to-speech pricing to cost per hour of input audio, and the current spread runs from about $1.42/hour at the cheap end to $10.75/hour for the most expensive premium tier, with a well-regarded mid-market option like Grok Voice Think Fast 2.0 landing at $4.80/hour — that is $0.08 per audio minute.

Read those two paragraphs together and the self-hosting case is weaker than it looks. A dedicated H100 is only cheaper than a mid-priced hosted endpoint if you keep it genuinely busy, and it is more expensive than the cheap end of the hosted market almost regardless of utilization — while you also absorb the engineering, the on-call, and a model that scores 29.2% where the hosted options score 87–96%.

And then there is the wall. The license is the OpenMDW License Agreement v1.1, and the card states in its own block quote that the model "is ready for research purposes only." The code on the GitHub branch is Apache-2.0; the weights are not. You can download this, study it, fine-tune it, benchmark it, and write about it. You cannot put it in front of customers. Every economic argument above is therefore academic for a company trying to ship a voice product — the question was never whether the GPU math worked.

Which is also why we will say the obvious thing plainly: NemotronLabs VoiceChat 11B is not callable through OrcaRouter, because it is not callable through anything. What one key does get you is the baseline it should be measured against — the hosted voice and audio models sit behind a single API at 0% markup, meaning we pass the provider's list price straight through, so when a vendor cuts its audio rate the lower number is what you pay that day rather than after a contract cycle. If you are pricing a voice agent, that per-minute figure is the number an 80 GB GPU has to beat, and it is worth knowing precisely before you commit to a rack.

The name problem: 11B, 12B, NemotronLabs, Nemotron 3

This is where most of the early coverage has gone wrong, so it is worth being careful about what is established and what is not.

Four of NVIDIA's own surfaces describe this work under three different labels. Hugging Face publishes "NVIDIA NemotronLabs VoiceChat 11B" with open weights and a research-only license. NVIDIA's developer blog, in March 2026, described "Nemotron 3 VoiceChat" as a 12B-parameter full-duplex model in early access, targeting sub-300 ms end-to-end latency on 80 ms chunks, with an early-access program offering reference containers, benchmark results and a fine-tuning path, and promising "open, inspectable weights for enterprise deployment." The public VoiceBench leaderboard and Artificial Analysis both file their entries as "Nemotron 3 VoiceChat (V1)," 12B.

What is knowable: the architecture descriptions match component for component — Fast Conformer encoder, Nemotron Nano v2 9B backbone, NVIDIA text-to-speech decoder, separate tool-calling channel, 80 ms frames, one unified stack. The unrendered card in the Hugging Face repo uses the 12B figure the other surfaces use. It is the same line of work, and the third-party entries are almost certainly measuring this family.

What is not confirmed: that the checkpoint you can download today is bit-for-bit what Artificial Analysis and VoiceBench evaluated, or what the early-access program hands out. Two details argue against assuming it. The parameter counts differ, 11.095B measured against 12B as filed. And the latency targets differ sharply — the blog's enterprise pitch is sub-300 ms end to end, while the shipped card measures 448 ms and 480 ms on Full-Duplex-Bench. Those could be different measurement points on the same model, or different models. NVIDIA has not said, because NVIDIA has not said anything about this release at all.

The practical upshot: if you cite a number for this model, cite which surface it came from. Coverage that attributes Artificial Analysis's 29.2% to the Hugging Face model card, or NVIDIA's 37.0% to an independent test, has merged two things that NVIDIA itself keeps in separate documents with different parameter counts.

Where it breaks

The draft card's limitations section is unusually candid, and the GitHub branch adds more. Collected:

• English only, and no multilingual roadmap in the repo.

• Two-minute memory. Conversation beyond a two-minute audio window may not be retained — a hard ceiling for support calls or anything that needs to remember what was agreed four minutes ago.

• Dumber than its own backbone. NVIDIA states it may underperform Nemotron Nano 9B v2 on knowledge, instruction-following and safety, because it was tuned for conversational naturalness. It was not explicitly trained for reasoning or alignment; multi-step reasoning and arithmetic are called out as weak.

• No reliable backchanneling. The "mm-hm" that signals a human is still listening is not systematically handled.

• It will cut you off mid-sentence. The branch notes list interrupting the user on a mid-sentence pause, and "runaway continuation / self-talk," among known failure modes — the direct cost of an encoder with zero right context.

• Quiet rooms only. Explicitly unsuited to noisy or reverberant environments, especially where background speech occurs. That rules out most call-center floors and most cars.

Who should actually download this

Researchers working on duplex speech modeling get the most here, and should move: an 11B open checkpoint with a working tool-calling channel, trained on roughly 550,000 hours of blended real and synthetic audio, is a far better starting point than reimplementing from the SALM-Duplex paper. The research license is not a constraint on that work.

Teams building voice agents should read the card and skip the download. Nothing you learn from a 44 GB float32 checkpoint you cannot ship will change your architecture, and the honest lesson of the benchmarks is that end-to-end full duplex is not yet competitive with a good hosted model on the thing users notice most, which is whether the assistant understood them. In the meantime the sensible move is to build against a hosted endpoint and keep the swap cheap — one key across 200+ models with automatic failover means a provider incident does not take your phone line down mid-call, and it means moving to an open full-duplex model later is a config change rather than a rewrite.

Enterprises who care about this specifically should apply to the Nemotron 3 VoiceChat early access rather than build on the Hugging Face weights. That program is where the deployable license, the reference containers and the sub-300 ms claim live. The public checkpoint is a research preview of the same idea, published without the paperwork that would let you use it.

Three questions this repo will keep getting

Can I use it commercially if I fine-tune it heavily? No. OpenMDW v1.1 plus the card's "research purposes only" statement governs the weights and anything derived from them; the Apache-2.0 license on the GitHub branch covers the inference code, not the checkpoint. Fine-tuning does not launder a license. If commercial rights are what you need, the early-access program is the route NVIDIA has actually set up for that.

Is this the same model as the one on the leaderboards? Same family, near-certainly; identical artifact, unconfirmed. The leaderboard and Artificial Analysis entries are filed as "Nemotron 3 VoiceChat (V1)" at 12B, the downloadable checkpoint measures 11.095B, and NVIDIA has published nothing reconciling them. Use the leaderboard numbers as the best available independent read on the lineage, not as a verified measurement of the file on Hugging Face.

Will it run on something smaller than an 80 GB card? Probably, with work, and nobody has published proof. The weights are float32, so a bfloat16 conversion should bring roughly 44 GB of parameters down to roughly 22 GB, which leaves real headroom on a 48 GB card before you account for the KV cache, the codec and the streaming buffers. But the supported path is vLLM on an H100 at 80 GB, the deployment container is built for that, and the only tested configuration NVIDIA reports is that one. Budget time, not just VRAM.

What to watch

Three things would each change the read on this release. A tech report, or even a card that stops contradicting itself, would tell us whether the 11B checkpoint is the artifact the leaderboards measured. An independent latency reproduction — someone outside NVIDIA confirming 448 ms turn-taking on their own hardware — would turn the release's most attractive claim from marketing into fact. And a commercial license, or a hosted endpoint from anybody, would move this from a paper-adjacent curiosity to a real option for the many teams who would happily trade some reasoning quality for a voice agent that can be interrupted.

Until one of those lands, the accurate description is narrow but genuinely notable: NVIDIA has published the strongest open full-duplex voice checkpoint yet measured, given it the first working tool-calling channel in that category, licensed it so that nobody can ship it, and declined to mention that any of this happened.