AI Agent Engineer

AI Agent Engineer
PERSONAL PROGRESS

Mark this course

Course content is public. Sign in to save personal progress.

Synthetic Data

5 min read

Synthetic Data

About this knowledge base

Synthetic data is new data constructed through rules, simulators, statistical models, or generative models. It can fill rare conditions, create controlled tests, reduce manual drafting effort, and support training or privacy research. It is not automatically realistic, unbiased, private, free of copyright risk, or fit for the target task.

This knowledge base follows synthetic evaluation data for AI Agent engineering: define purpose and data contract first; use templates and models to cover conditions; then filter, deduplicate, split by family, calibrate against real data, review risk, and publish a versioned dataset. Its core principle is: generation only produces candidate samples; independent validation supplies quality evidence.

The course covers four method classes: template/combinatorial generation, state machines/simulators, statistical synthesis, and model generation. Their guarantees differ. Templates are easy to verify but offer limited linguistic variation; simulators produce trajectories and terminal states but depend on an environment model; statistical synthesis can approximate selected statistics but does not automatically protect privacy; generative models expand open-ended expression but introduce label, memorization, bias, and source risks. Choose the purpose before choosing a popular tool.

Important boundary

“Synthetic” is not a privacy guarantee. A generative model may reproduce training content, and statistical synthesis based on sensitive originals may leak too. Only a clearly specified, correctly implemented, and audited mechanism—such as differential privacy—provides its corresponding guarantee. Qualified people in the organization must assess law, license, and personal-information handling against the data source and jurisdiction.

Position in the overall route

This course belongs to “Production, Evaluation, and Governance.” Learn Data Cleaning, Data Annotation, and Evaluation Framework first. Then use it to add stress layers for Benchmark Design or prepare candidate data for model/Agent training.

Learning objectives

  • Write data contracts and stop conditions for evaluation, training, simulation, or privacy-release purposes.
  • Generate reproducible, traceable samples from templates, factor matrices, and fixed seeds.
  • Design model-generation pipelines that control condition quotas rather than blindly maximizing count.
  • Perform Schema/rule/quality/exact-or-near-duplicate filtering and family-level splits.
  • Evaluate fidelity, coverage, diversity, downstream utility, and privacy risk independently.
  • Distinguish sample validity, dataset coverage, held-out-real-data utility, and formal privacy guarantees rather than substituting one aggregate score for another.
  • Recognize memorization leakage, bias amplification, copyright/license risk, and synthetic-data contamination.
  • Publish maintainable datasets through Data Cards, content fingerprints, versions, and changelogs.
  • Record sample-level provenance, source/authorization evidence reference, access scope, and withdrawal impact separately; none of those fields alone is a legal or privacy conclusion.

Prerequisites

  • Read and write JSON and understand train/dev/test and group splits.
  • Run Python 3 standard-library scripts in PowerShell 7.
  • Understand basic proportions, distributions, and sampling; no deep-learning training is required.
  • Know that model-generated fluent text still needs validation and cannot be treated as a correct label.
  1. Purpose and Data Contract — define use, population, Schema, and acceptance first.
  2. Template and Programmatic Generation — build a deterministic, inexpensive, coverable baseline.
  3. Model Generation and Condition Coverage — expand linguistic and task variation under strict provenance.
  4. Filtering, Deduplication, and Group-Level Splits — turn the candidate pool into usable data.
  5. Quality, Utility, and Real-Data Calibration — show that data helps the target decision.
  6. Privacy, Memorization, Bias, and Copyright — identify risks generation cannot remove.
  7. Versioning, Evaluation, and Release — record lineage, limits, and updates.
  8. Project: Offline Synthetic Evaluation Data Pipeline — run condition matrix, filtering, deduplication, split, and fingerprint.

Hands-on entry point

  • Main project: Offline Synthetic Evaluation Data Pipeline. It uses a strict JSON specification, condition matrix, source declaration, rejection log, family split, and quality gate. It supplies executable PASS, BLOCK, and contract-error cases plus 43 tests.
  • Design task: begin with one production failure, write three real variants and six synthetic stress variants, and state their different weight in reporting.
  • Audit task: write provenance, model/template version, filtering record, and known risks for any synthetic dataset.

Mastery criteria

  • I can explain why “more synthetic samples” is not “more useful information.”
  • I can define coverage targets with a task-factor matrix and retain family IDs.
  • I can distinguish Schema validity, semantic correctness, realism, diversity, and downstream utility.
  • I can do normalized exact deduplication and explain when near-duplicate or semantic review is needed.
  • I can test training or evaluation utility against an independent real holdout.
  • I can state the guarantee difference between ordinary synthetic data and differentially private synthesis.
  • I can publish a Data Card with version, hash, source, license, risk, and limitation.
  • I can run the offline project and its 43 three-mode tests on Windows 11 / PowerShell 7, and explain why PASS still does not prove production utility or anonymity.

Relationships to other knowledge bases

  • Data Cleaning supplies Schema, dirty-value, and anomaly handling.
  • Data Annotation owns rubrics, human calibration, and disagreement resolution.
  • Evaluation Framework decides how synthetic cases are scored; synthetic data cannot be its own only validator.
  • Benchmark Design owns frozen splits, contamination control, and comparison protocol.
  • Privacy Computing studies formal techniques such as differential privacy in more depth.
  • AI Governance owns authorization, data responsibility, release approval, and risk acceptance.

Primary references

The following official material, standards, or original papers were retrieved/checked on 2026-07-22:

Backlinks