Agent Model: Nemotron - Data Recipe

There is a lot of material here. Across most industry practice, the thinking behind the Data Recipe is broadly similar. The release includes many open-source Pipeline implementations and ideas for designing Data at different Stages, with plenty worth learning from and applying. Link: Nemotron

Data Library

This is the Data processing logic shared across the models in Nemotron.

Pretraining

Pretraining uses similar Pipelines for different data sources. These cover both pretraining data processing and sample preprocessing:

1. Data Sources: Common Crawl / Github / PDF / HF datasets / Internal Data

2. Extraction -> Cleaning -> Deduplication -> Quality Classification -> Synthesis/Rewriting

Take Nemotron-CC’s production Pipeline for Web-text pretraining as an example:

1. After Common Crawl -> use JustText to extract the main text
2. Use FastText for language identification and remove data with high perplexity (minimum confidence: 0.3)
3. Exact Document Dedup (deduplicate using the MD5 of the main text/url)
4. MinHash/LSH fuzzy Dedup (the MinHashLSH algorithm discussed earlier; defaults to char 24-gram, 20 bands, and 13 hashes per band, processing 5 bands at a time for deduplication)
   • See: https://www.liaojiayi.com/blog/llm-text-dedup
5. Exact Substring Dedup uses Google’s deduplicate-text-datasets here (an approximate deduplication method based on matching repeated prefixes), processing 100GB/chunk
6. Quality classification: use three classifiers—FineWebNemotronEdu, FinWebMixtraEdu, and fasttext-oh-eli5—to assign quality scores, dividing the 5-95 percentile range into 20 buckets. (These include open-source and fine-tuned models; see the fineweb-edu-classifier recipe.)
7. Apply SDG (Synthetic Data Generation) to high-quality data, taking only the highest-quality data from buckets 18/19 above to produce Diverse-QA data, Knowledge, and more, using Teacher models such as Qwen3-30B-A3B;

3. nemotro.data_prep (Sample Preprocessing)

Flow: Plan -> Download -> Tokenize / Pack / JSONL shard. This process does not inherently need to be so complex; some computation could even be offloaded to Torch’s DataLoader. In practice, however, many companies do this to improve training efficiency on massive datasets by reducing GPU waiting. (For example, ByteDance: Teaching the Old Dog New Tricks: Building Efficient Data Pipelines for Large-Scale LLM Pre-Training (Operational Systems) ).

• Plan: divide the raw data into shards according to the training strategy, effectively sharding the data in advance;
• Download: download by shard (primarily for HF Datasets here)
• Tokenize / Pack / JSONL shard
  • Tokenize the data directly and store it in .bin (the tokenized binary file) and .idx (the index for each row). Random reads during training can then locate the corresponding data directly. Finally, output a blend.json, for example:

1
{
2
  "datasets": [
3
    {
4
      "name": "my-dataset",
5
      "path": "hf://org/dataset",
6
      "subset": "config-name",
7
      "split": "train",
8
      "weight": 2.0
9
    }
10
  ]
11
}

4. Megatron-Bridge: Read Blended Data During Training, Sample Fixed-Length Sequences, rollout

Model Data Recipe

Next, let’s look at the specific Data Recipe for each model.

Note: B = 1 billion parameters; A denotes the number of parameters activated per token. Model names use rounded nominal figures; for example, Nano’s official model card lists approximately 3.5B active parameters.

Nemotron 3 Nano 30B-A3B

Pretraining

A total of 25T tokens is used, split into two phases:

• Phase 1: 23.5T, emphasizing sources and diversity
• Pase 2: 1.5T, providing high-quality, high-quality synthetic, and STEM/specialized data, while removing low-quality Web data

An overview of NVIDIA’s own data families:

Mid-Training / SFT

• Long-Context Mid-Training: after the main Pretraining phase, use 121B tokens to extend the context to 1M. The aim is not another round of broad knowledge acquisition, but to adapt the Model to very long inputs, while mixing in high-quality short examples to prevent drift.

• SFT: covers downstream subtasks such as Math, Code, STEM QA, and Tool Use. High-quality data teaches the range of capabilities built on top of the Pretraining base model in one stage, preparing it for subsequent RL.

RL

The complete approach includes three categories:

1. Multi-environment RLVR/GRPO: train a Batch across 6 environments: math_with_judge, code_gen, mcqa, instruction_following, workplace_assistant, and structured_outputs_json. The environments use judging against reference answers, option matching, constraint verification, task completion, and other signals as rewards;
2. GenRM RLHF: 16 responses per prompt, with circular comparison reducing pairwise comparison complexity from O(N^2) to O(N), plus length and format penalties;
3. DPO for tool hallucination: use perference pairs to reduce unnecessary tool calls, applying targeted behavioral corrections after the main RL pipeline;

In addition, apply strip reasoning (remove reasoning) to 10% of samples, and truncate 3% of reasoning traces at randomly chosen budgets. This trains the model to function normally even when reasoning is disabled or the token budget is limited. (Not all data needs to target long thinking.)

Nemotron 3.5 Lightning 30B-A3B

NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents

• This is NVIDIA’s model specifically for Long-Running Agents

Pretraining

The Text data reuses several of Nano’s large datasets and its approach of diversity first, then high quality. Overall, it is still quite similar to Nano.

RL

For the Agent side, RL deserves particular attention. Like Nano, it follows similar GRPO, GenRM, and DPO Recipes. The differences are:

Agentic Terminal Pivot

This is a data construction approach that splits long trajectories into many training samples centered on intermediate decision points. See NVIDIA’s Agentic-Terminal-Pivot dataset. Its data mainly covers Terminal-related tasks, similar to Terminal-Bench. GLM-5.1 generates trajectories, and valid assistant turns are then split into multiple samples.

Example:

Complete trajectory:
Task → Inspect directory → Read code → Run tests → Edit code → Verify success

Extracted decision points:
Input: Task + directory-inspection history and results
Reference: Read code

Input: Task + prior history + test failure information
Reference: Edit code

Input: Task + prior history + edit results
Reference: Run tests

PTQ(Post-Training Quantization) + QAD(Quantization-Aware Distillation)

Flowchart:

      Trained BF16 model
               │
               ├────────────────────────────────────────────┐
               ↓                                            ↓
PTQ: Low-precision quantization                     Keep BF16 teacher
               ↓                                    Freeze parameters
       Quantized student                                    │
               │                                            │
               └ QAD: Align output distributions ───────────┘
                                     ↓
                        Export low-precision model

Nemotron 3 Super 120B-A12B

Super is positioned as a large model for complex scenarios: multi-Agent collaboration, enterprise process automation, IT ticket handling, complex coding tasks, and multistep tool use.

Pretraining

The pretraining Token count is also 25T, the same as Nano, but the model size increases from 30B -> 120B. The data mixture is therefore worth a closer look.

The main difference is Phase2’s 5T of data, substantially more than Nano’s 1.5T. Let’s look at the differences in Phase2 data:

Note that not all high-quality data is upweighted. Some synthesis data is actually downweighted, with the budget going to FinePDF and specialized data. This suggests that Phase2 aims to improve the final distribution of capabilities, rather than maximize a single quality score.

Long-Context Mid-Training

Add Nemotron-Pretraining-Long-Context-v1 to the Phase 2 mixture (I can no longer find this dataset). It accounts for 20%, with the remaining 80% coming from P2’s high-quality data.

• Stage 1: 34B tokens, 1M context
• Stage 2: 17B tokens, alternating between 1M context and 4K context to recover Math-related Benchmark regressions caused by 1M context training. This shows just how important Benchmarks are.

SFT

The SFT data improves on Nano’s datasets:

1. Reuse: Chat, InfiniByte, formal proofs
2. Rebuild with different Teacher Models: competition mathematics, multilingual data, and tool use, using DeepSeek-v3.2/Kimi K2 and others
3. Add: SWE, search, Terminal, Long Context, and more

RL

Super focuses on long-horizon tasks, making its RL more rigorous and complex than Lightening’s, with a strict 6-stage curriculum:

1
SFT
2
|
3
RLVR1 -> RLVR2 -> RLVR3
4
|
5
SWE1 -> SWE2
6
|
7
GenRM RLHF

• RLVR1-3: 21 reward environments and 37 datasets. First filter out prompts that are too easy: run them with the SFT checkpoint and discard those with excessively high pass rates. Then switch blend.json from easier to harder mixtures, training math/code/MCQA/instruction following/tool/structured output. Add low-effort reasoning as well: no excessive thinking when reasoning is unnecessary!
• SWE1: a more general software-engineering pivot that familiarizes the model with the loop of repo browsing, editing, execution, and testing
• SWE2: sandbox environments and complete execution for SWE benchmarks, using actual execution results for RL
  • This is also an advantage over Lightening: e2e software-development RL. Lightening focuses more on step-by-step learning with simpler tasks.
• RLHF: GenRM circular comparison + KL/length control to correct style, length, and other formatting issues

Further RL Enhancements in Nemotron 3 Ultra 550B-A55B

It is worth highlighting the RL optimizations in another, larger model: Nemotron 3 Ultra 550B-A55B.

Ultra adds MOPD, multi-teacher on-policy distillation:

• Train more than ten domain-specific teachers.
• Ultra generates its own responses or action trajectories. This is the fundamental difference from distillation: the student performs its own rollout, then moves closer to the teacher model.
• The corresponding teacher provides token-level guidance.
• Perform two rounds of iterative teacher-student training.

Nemotron 3 Nano Omni

This is the Omni model covering all modalities, and it is interesting too (see https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Omni-report.pdf). It involves alignment across multiple modalities. Training any one modality can degrade capabilities in others, making the training process quite complex:

Existing language model + existing vision encoder + existing audio encoder
  → Vision projector alignment
  → Joint vision-language SFT
  → Audio projector alignment
  → Audio encoder and projector training
  → 16K joint all-modality SFT
  → 48K joint long audio/video SFT
  → 256K long-document SFT
  → MPO → Text RL 1 → Image RL → Omni RL → Text RL 2

Pretraining

• Stage0: train the Vision Projector. First freeze the backbone and encoders, so that the vision projector learns to map image features into the language model’s input space.
• Stage1: expand the scope of training to visual question answering, documents, charts, GUI, and visual reasoning.
• Stage2: use Granary v1.1 ASR data to align the audio projector.
• Stage3: Stage 3 unfreezes the audio encoder and expands into four categories of audio tasks.
• Stage4: train all parameters across vision, text, audio, video, joint audio-video question answering, and safety tasks. According to the token proportions in Table 3 of the paper, vision accounts for 53.4%, general audio for 16.9%, short video for 11.0%, and text for 6.1%; the rest includes reasoning, ASR, Omni, and safety data.
• Stage5: extend Context to 48K and reduce the proportion of short samples.
• Stage6: focus on Long Context for long texts, which is no different from the LLM approach above.

RL

Omni’s RL sequence:

1. MPO preference optimization: OpenGVLab/MMPR, first improving visual response quality and preference alignment;

• OpenGLLab/MMPR is a multimodal reasoning preference dataset,
• MPO is Mixed Preference Optimization, which includes DPO and BCO (Binary Classifier Optimization, labeling responses as good or bad)

2. Text RL: reuse Nano’s RL Blend;
3. Vision RL: OpenGVLab/MMPR-Tiny, using verifiable visual tasks to improve the model’s Grounding capability;

• For example, answering questions based on images