Skip to content

Quick Start

Evaluate a synthetic dataset in 5 minutes.

1. Prepare Your Data

You need:

  • A real CSV file (your original dataset)
  • A synthetic CSV file (generated by any method)
  • Both must share the same columns

2. Create a Configuration File

Create config.yaml:

data:
  real: "data/real/my_dataset.csv"
  synthetic: "data/synth/my_synthetic.csv"
  target: "label_column"          # target for ML metrics (or "None")
  task_type: "classification"     # classification | regression | "None"
  schema:
    id_col: id                    # excluded from analysis
    age: continuous
    gender: categorical
    income: continuous
    label_column: categorical

metrics:
  - "fidelity"                    # all 26 fidelity metrics
  - "utility.tstr"               # Train-Synthetic, Test-Real
  - "privacy.dataset_based"      # DCR, NNAA, MIA, k-Anonymity, ...

calibration:
  n_iterations: 5

report:
  formats: ["json", "md"]
  output_dir: "reports/"

3. Run Evaluation

metis evaluate --config config.yaml

4. Review Results

reports/
├── summary.json       # Structured results
├── all_metrics.json   # Detailed per-metric data
└── summary.md         # Human-readable report

The summary.json contains:

{
  "scores": {
    "fidelity_score": 0.82,
    "utility_score": 0.78,
    "privacy_score": 0.65,
    "composite_score": 0.75
  }
}

5. Use the Python SDK

from metis import evaluate_from_config

summary = evaluate_from_config("config.yaml")
print(f"Composite: {summary.aggregates['composite_score']:.2f}")

Next Steps