L7-08 · Reproducible quantum causal benchmarks

Learn¶

A benchmark needs an explicit data generator or physical source, train and test splits, metrics, and resource limits. For causal tasks, reserve intervention tests that differ from training conditions. Predictive agreement on the original settings does not by itself measure intervention recovery.

Compare with strong classical models, simple physical baselines, and ablations. An ablation removes a selected feature to test its contribution. Match tuning budgets and prevent test-set feedback during model selection. Use several seeds or repeat experiments to quantify variation.

Record the exact circuit, device calibration, software version, shot allocation, and stopping rule. Save raw counts where permitted and enough metadata to reconstruct the analysis. Separate simulator results from hardware results. A simulation validates code and model calculations, not an advantage on a physical processor.

Practice¶

Check your understanding¶

Which split tests intervention recovery more directly?

• A. Another copy of the same training records

• B. Held-out informative intervention settings

• C. A larger training score display

Sources¶

This lesson draws on Huang et al., 2021Costa & Shrapnel, 2016Biamonte et al., 2017.

Next: L7-09 · A research protocol from question to result