What Makes Good Data Judgment?

To make data taste in LLM development concrete, consider the complete data loop: collection, selection, use, quality assessment, and evaluation.

Introduction

What Makes a High-Quality Training Dataset for Large Language Models: A Practitioners’ Perspective

• This 2024 study interviewed 219 LLM practitioners across countries about training-data quality:
  • Top three qualities: Reliability of the source, relevance to the goal, and accuracy without obvious errors.
  • Top three validation methods: Manual sampling, quantitative metrics, and visualization.
  • Challenges: No standardized processing pipeline, uncertainty about how much cleaning is enough, and difficulty quantifying cleaned-data quality.

There is no standard answer to what makes good LLM data. People solve these problems using their own judgment, or data taste.

Training Compute-Optimal Large Language Models:

• DeepMind’s classic scaling-law paper suggests that under a training-compute budget, parameter count (N) and training tokens (D) should grow roughly together as (C^{0.5}).
• With similar training compute, Chinchilla’s 70B parameters and 1.4T tokens outperformed Gopher’s 280B parameters and 300B tokens, while making inference cheaper.

Chinchilla showed that more data can raise model performance and that larger models can absorb more data. Architectural progress has continued increasing how much data a given model or compute budget can use. The lesson for foundation-model companies was clear: do not underestimate model capacity; find more data.

Finding Good Data

Without online feedback or purchased datasets, model data commonly comes from the following sources:

• Crawling: Most foundation-model data comes from crawls. Quality varies widely, requiring extensive cleaning. Two classic papers:
  • https://arxiv.org/abs/1911.00359: Uses FastText classification and confidence filtering on crawled text, with Wikipedia KenLM for perplexity stratification.
  • The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale: Found that aggressive filtering and deduplication of raw crawls could reduce experimental performance.
• Open datasets: Hugging Face is useful; even foundation-model developers such as Kimi use its datasets as seeds in new domains. Quality is often higher than raw crawls but still imperfect, especially when datasets combine and synthesize others and inherit LLM rewriting errors. Data-focused companies should:
  • Maintain a daily Hugging Face monitor for relevant domains.
  • Check provenance first. A reputable company or lab often provides an initial quality signal.
  • Examine coverage, validity, formats, and whether processing code is available.
• Synthesis: When desired data is scarce, such as high-quality or task-specific examples, generate it through approaches including:
  • Distillation: Stealing Reasoning Traces from Proprietary LLM APIs describes a recent method for distilling Opus models; the vulnerability has since been closed.
  • Agent-driven generation: As discussed below, a harness guides an agent through a simulated environment, then rejection sampling retains high-quality trajectories.
  • Rewriting: For example, generate cross-context questions and answers from long documents to train long-context ability. Kimi 3 emphasizes extensive rewriting with a state-of-the-art model to reuse valuable long documents.
  • Modality conversion: Images or videos converted to text can reveal information absent from ordinary webpages, such as mathematics textbooks.

Selecting Good Data

Selection matters after collection. I recently saw a post claiming that the once-popular data-selection field had disappeared: as models scale, filtering matters less, and excessive filtering distorts distributions and hurts performance.

From my perspective, selection remains crucial, especially for startups trying to initialize a foundation model cheaply. Finding critical, high-quality data is their most important problem.

Later, when more high-quality data is unavailable and deeper cross-domain relationships matter, using diverse and lower-quality data becomes unavoidable.

• What Makes Good Data for Alignment? :
  • Meituan’s 2024 paper combines complexity, quality, and diversity. GPT creates evolutionary sequences in the DEITA pipeline to generate better data and improve models.
  • Experiments used 7B/13B models, so conclusions may not hold at today’s larger scales.

• Large-Scale Data Selection for Instruction Tuning :
  • A 2025 paper found that high-quality selection methods such as top-K can fail with millions or tens of millions of source examples, potentially due to bias or judge-model preferences. RDS+ represents data through hidden states, perhaps more naturally than external filters. However, representations differ across layers, so I suspect stability remains an issue.

ATLAS

• Google’s 2024 paper further supports the importance of selection, with several findings:
  • Cross-language transfer is directional. Spanish may help Catalan, but A helping B does not imply B helps A. Identify which source tokens provide the greatest marginal benefit.
  • Multilinguality consumes capacity, but positive transfer offsets some cost. Doubling languages from K to 2K requires about 1.18x parameters, 1.66x data, and 1.96x compute tokens, while retaining only 83% of original-language data to maintain its loss.
  • Low-resource languages cannot benefit indefinitely from repetition: after one epoch, returns decline rapidly.

Using the Right Data at the Right Time

Pretraining, SFT, and RL generally need different data:

• Pretraining: Broad, diverse data builds language, knowledge, and foundational abilities.
• SFT: High-quality instruction-answer demonstrations covering target tasks teach instruction following, response formats, and typical solutions.
• RL: Feedback from models or human labels improves alignment with experts and draws out more of the potential learned during pretraining.

DeepSeek-R1: Pure-RL R1-Zero suffered from poor readability and mixed languages.

• R1 obtains high-quality SFT data through careful filtering, model judgment, and human judgment.
• RL rewards for accuracy, format, length, and language further improve chain-of-thought trajectories.

Nemotron

NVIDIA’s open Nemotron recipe analyzes agent-model training in detail:

• Pretraining: 23.5T tokens spanning web, code, academic, multilingual, and other domains.
• SFT: 1.5T tokens of high-quality, synthetic, STEM, and textbook material.

Different abilities require different data at different stages:

1. Long-context ability: CPT: Training sequences up to 512k. SFT: Long contexts averaging 128k, with a 256k maximum.

Many controllable synthetic tasks are used, for example:

• Place required information far apart.
• Distribute evidence across documents.
• Add irrelevant distractor documents.
• Require cross-document, multihop answers.
• Retain automatically verifiable reference answers.

2. Agent ability At SFT, the objective extends beyond pretraining’s p(next token|previous tokens) to agent behavior, multiple turns, and reasoning, essentially training P(action|state).

3. Reasoning ability The recipe also incorporates reasoning control into training data:

• Remove reasoning from about 10% of examples to train reasoning on/off control.
• Truncate about 3% of reasoning trajectories to different token budgets.
• At inference, insert when the budget is reached, requiring a final answer from incomplete reasoning. Reasoning budgets are therefore not purely an inference feature. Models must first see examples of:
• Direct answers without reasoning.
• Short reasoning.
• Long reasoning.
• Concluding an answer after reasoning is interrupted.

A Contrarian View: Can Dirty Data Be Valuable?

• A Bitter Lesson for Data Filtering argues that some dirty data is acceptable. Two main experiments:

1. Apply language, interjection, deduplication, and other filters to obtain cleaner data; at the same model scale, learning was less complete than with the full dataset.
2. Inject low-quality shuffled data; the model still learns some word relationships from it.

The paper’s framing may be limited by its assumption that high-quality data is scarce. In practice, teams continually create more, including synthesis from lower-quality sources.

Assessing Data Quality

DataDecide: How to Predict Best Pretraining Data with Small Experiments:

• A 2025 paper asks whether small models can cheaply predict better training datasets. Key findings:
  • Rankings from a 150M model predict roughly 80% of pairwise dataset choices for a 1B model.
  • Use answer likelihood from logits rather than accuracy. On MBPP/HumanEval, small models may fail to solve tasks while answer likelihood still carries useful signals, bringing ranking accuracy to about 80%. To me, this also suggests insufficient compute budget.

This makes sense: fully pretraining a base model on every dataset variation is impractical, so a reliable baseline matters.

DataMan: Data Manager for Pre-training Large Language Models:

• Alibaba’s 2025 paper uses GPT-4 to score 35,700 documents across thirteen dimensions, then fine-tunes Qwen2-1.5B into DataMan. LLaMA experiments show that training on scores 4/5 substantially outperforms random sampling.

Fine-tuning a private-domain evaluator seems reasonable. Kimi also uses this extensively in its data pipeline.

DataRater: Meta-Learned Dataset Curation:

• A 50M-parameter Transformer scores data during training, using the original model’s loss as the target.
• New data passes through the scorer, and low-scoring examples are discarded, effectively selecting examples expected to produce larger gradients.

The idea is attractive but problematic. Models differ, large-model effects may not map to small-model losses, and larger gradients are not necessarily better.

Synthetic Data: High Quality at Lower Cost

Public agent datasets are scarce, so most long-horizon agent training data is collected through synthesis.

Kimi K3: Open Frontier Intelligence: For Kimi K3, the report describes building an agent-task knowledge graph from internet information, repeatedly discovering potential tasks and trajectories with deduplication and concept merging.

Qwen-AgentWorld: Language World Models for General Agents

• Qwen’s general-agent paper explains using seed data to synthesize large numbers of trajectories.

WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents

• For WeChat multimodal search, WeAgent improves image understanding through reverse generation. Start with an internet question that requires an image, then extend it backward into executable multihop tasks. This yields long trajectories, from which rejection sampling retains high-quality examples.

Evaluating Data

Understanding the Dataset Practitioners Behind Large Language Model Development:

• Google Research examined this problem in 2024. Different model-team roles focused on different concerns, such as benchmark results or real user distributions, but their methods were similar:
  • Eyeballing: Inspect examples and judge intuitively. Samples are often unrepresentative, and people may look for evidence supporting a preexisting conclusion.
  • Notebooks: Analyze distributions and other properties, but highly customized work makes consensus and standardization difficult.

This remains a painful development problem without an obvious solution. As internet products evaluate changes through A/B metrics, model iteration needs a comprehensive, reliable experimentation platform.