NVIDIA AI Releases Nemotron-Elastic-12B: A Single AI Mannequin that Provides You 6B/9B/12B Variants with out Additional Coaching Price

Why are AI dev groups nonetheless coaching and storing a number of giant language fashions for various deployment wants when one elastic mannequin can generate a number of sizes on the similar value? NVIDIA is collapsing the standard ‘mannequin household’ stack right into a single coaching job. NVIDIA AI workforce releases Nemotron-Elastic-12B, a 12B parameter reasoning mannequin that embeds nested 9B and 6B variants in the identical parameter area, so all three sizes come from one elastic checkpoint with no further distillation runs per measurement.

Many in a single mannequin household

Most manufacturing methods want a number of mannequin sizes, a bigger mannequin for server facet workloads, a mid measurement mannequin for robust edge GPUs, and a smaller mannequin for tight latency or energy budgets. The standard pipeline trains or distills every measurement individually, so token value and checkpoint storage scale with the variety of variants.

Nemotron Elastic takes a special route. It begins from the Nemotron Nano V2 12B reasoning mannequin and trains an elastic hybrid Mamba Consideration community that exposes a number of nested submodels. The launched Nemotron-Elastic-12B checkpoint might be sliced into 9B and 6B variants, Nemotron-Elastic-9B and Nemotron-Elastic-6B, utilizing a supplied slicing script, with none further optimization.

All variants share weights and routing metadata, so coaching value and deployment reminiscence are tied to the most important mannequin, to not the variety of sizes within the household.

Hybrid Mamba Transformer with elastic masks

Architecturally, Nemotron Elastic is a Mamba-2 Transformer hybrid. The bottom community follows the Nemotron-H model design, the place most layers are Mamba-2 primarily based sequence state area blocks plus MLP, and a small set of consideration layers protect world receptive subject.

Elasticity is applied by turning this hybrid right into a dynamic mannequin managed by masks:

• Width, embedding channels, Mamba heads and head channels, consideration heads, and FFN intermediate measurement might be lowered by binary masks.
• Depth, layers might be dropped in accordance with a realized significance ordering, with residual paths preserving sign move.

A router module outputs discrete configuration decisions per finances. These decisions are transformed to masks with Gumbel Softmax, then utilized to embeddings, Mamba projections, consideration projections, and FFN matrices. The analysis workforce provides a number of particulars to maintain the SSM construction legitimate:

• Group conscious SSM elastification that respects Mamba head and channel grouping.
• Heterogeneous MLP elastification the place totally different layers can have distinct intermediate sizes.
• Normalized MSE primarily based layer significance to resolve which layers keep when depth is lowered.

Smaller variants are at all times prefix choices within the ranked element lists, which makes the 6B and 9B fashions true nested subnetworks of the 12B dad or mum.

Two stage coaching for reasoning workloads

Nemotron Elastic is skilled as a reasoning mannequin with a frozen trainer. The trainer is the unique Nemotron-Nano-V2-12B reasoning mannequin. The elastic-12B pupil is optimized collectively for all three budgets, 6B, 9B, 12B, utilizing information distillation plus language modeling loss.

Coaching runs in two phases:

• Stage 1: quick context, sequence size 8192, batch measurement 1536, round 65B tokens, with uniform sampling over the three budgets.
• Stage 2: prolonged context, sequence size 49152, batch measurement 512, round 45B tokens, with non uniform sampling that favors the complete 12B finances.

The second stage is vital for reasoning duties. The above desk exhibits that for AIME 2025, the 6B mannequin improves from 56.88 to 68.13, a 19.8 p.c relative achieve, whereas the 9B mannequin good points 9.7 p.c and the 12B mannequin good points 4.0 p.c after prolonged context coaching.

Finances sampling can be tuned. In Stage 2, non uniform weights of 0.5, 0.3, 0.2 for 12B, 9B, 6B keep away from degradation of the most important mannequin and preserve all variants aggressive on Math 500, AIME 2025, and GPQA.

Benchmark outcomes

Nemotron Elastic is evaluated on reasoning heavy benchmarks, MATH 500, AIME 2024, AIME 2025, GPQA, LiveCodeBench v5, and MMLU Professional. The beneath desk summarizes cross at 1 accuracy.

The 12B elastic mannequin matches the NanoV2-12B baseline on common, 77.41 versus 77.38, whereas additionally offering 9B and 6B variants from the identical run. The 9B elastic mannequin tracks the NanoV2-9B baseline intently, 75.95 versus 75.99. The 6B elastic mannequin reaches 70.61, barely beneath Qwen3-8B at 72.68 however nonetheless robust for its parameter rely provided that it’s not skilled individually.

Coaching token and reminiscence financial savings

Nemotron Elastic targets the fee downside straight. The beneath desk compares the token budgets wanted to derive 6B and 9B fashions from a 12B dad or mum:

• NanoV2 pretraining for 6B and 9B, 40T tokens complete.
• NanoV2 Compression with Minitron SSM, 480B exploratory plus 270B ultimate, 750B tokens.
• Nemotron Elastic, 110B tokens in a single elastic distillation run.

The analysis workforce studies that this provides round 360 instances discount versus coaching the 2 further fashions from scratch, and round 7 instances discount versus the compression baseline.

Deployment reminiscence is lowered as properly. The beneath desk states that storing Nemotron Elastic 6B, 9B, and 12B collectively requires 24GB of BF16 weights, whereas storing NanoV2 9B plus 12B requires 42GB. It is a 43 p.c reminiscence discount whereas additionally exposing an additional 6B measurement.

Comparability

Key Takeaways

1. Nemotron Elastic trains one 12B reasoning mannequin that incorporates nested 9B and 6B variants which might be extracted zero shot with out further coaching.
2. The elastic household makes use of a hybrid Mamba-2 and Transformer structure plus a realized router that applies structured masks over width and depth to outline every submodel.
3. The strategy wants 110B coaching tokens to derive 6B and 9B from the 12B dad or mum which is about 7 instances fewer tokens than the 750B token Minitron SSM compression baseline and about 360 instances fewer than coaching further fashions from scratch.
4. On reasoning benchmarks akin to MATH 500, AIME 2024 and 2025, GPQA, LiveCodeBench and MMLU Professional the 6B, 9B and 12B elastic fashions attain common scores of about 70.61, 75.95 and 77.41 that are on par with or near the NanoV2 baselines and aggressive with Qwen3-8B.
5. All three sizes share one 24GB BF16 checkpoint so deployment reminiscence stays fixed for the household in contrast with round 42GB for separate NanoV2-9B and 12B fashions which provides about 43 p.c reminiscence financial savings whereas including a 6B possibility.

Nemotron-Elastic-12B is a sensible step towards making reasoning mannequin households cheaper to construct and function. One elastic checkpoint produces 6B, 9B, and 12B variants with a hybrid Mamba-2 and Transformer structure, a realized router, and structured masks that protect reasoning efficiency. The strategy cuts token value relative to separate compression or pretraining runs and retains deployment reminiscence at 24GB for all sizes, which simplifies fleet administration for multi tier LLM deployments. Total, Nemotron-Elastic-12B turns multi measurement reasoning LLMs right into a single elastic methods design downside.

Take a look at the Paper and Model weights. Be happy to take a look at our GitHub Page for Tutorials, Codes and Notebooks. Additionally, be happy to observe us on Twitter and don’t neglect to affix our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.