ADACL Training Pipeline Explained Step-by-Step

Welcome to Lesson 34 of the SNAP ADS Learning Hub! Module 6 has built up the ADACL framework piece by piece — the motivation (Lesson 29), the binary-confidence problem it solves (Lesson 30), the framework definition (Lesson 31), intelligent augmentation (Lesson 32), and the continuous-labelling + CNN regression core (Lesson 33). This lesson wires those pieces into a single end-to-end training pipeline you could implement, run, and ship.

If the last five lessons were the what and why of ADACL, this one is the how-it-runs. By the end you should have a clear mental model of the data flow from raw sensor readings on a quantum device to a trained, calibrated, deployable anomaly detector.

Think of this lesson as the production pipeline diagram for an automotive assembly line. Each station does one well-scoped job, the output of one station feeds the next, and there’s a feedback loop at the end that catches defective output and routes corrections upstream. ADACL’s training pipeline has the same shape.

Pipeline Overview: Seven Stages

The pipeline has seven named stages, executed in order each training run:

1. Data ingestion — collect raw multi-modal signals + DeCoN-PINN predictions.
2. Feature engineering — derive the 394 physics-informed features.
3. Augmentation loop — generate synthetic examples spanning the drift space.
4. Continuous labelling — compute physics-informed deviation labels for every example.
5. CNN regression training — train the CNN regressor against the continuous labels.
6. Calibration & validation — calibrate scores against held-out operator-validated data.
7. Deployment handoff — package the trained model, calibration tables, and feature recipes for serving.

Each stage is examined below. Module 7 returns to several of these for deeper technical treatment.

Stage 1 — Data Ingestion

Inputs to the pipeline:

• Raw sensor streams: qubit measurements (single-shot readouts, T₁/T₂ estimates), control-pulse parameters (amplitudes, durations, phases), environmental sensors (refrigerator temperature, vibration, EMI).
• DeCoN-PINN predictions: the expected state evolution for the same time windows, as predicted by the upstream physics-preserving model.
• Operator-labelled positives: the (small) corpus of real anomaly events tagged by operators during normal operation of the system.
• Operator-labelled negatives: a sample of confirmed-normal time windows for calibration.

Ingestion synchronises these streams to common time windows, handles missing data, and produces a unified per-window record. Sloppy ingestion here ripples through every downstream stage; ADACL’s ingestion is strict (Lesson 37 covers the gory details).

Stage 2 — Feature Engineering

Each per-window record goes through a feature-engineering layer that produces ~394 physics-informed features (covered in detail in Lesson 37). The features fall into four families: statistical, quantum-specific, frequency-domain, and physics-constraint. Each is computed deterministically from the raw inputs and forms a column of the training tensor.

The output of this stage is a tensor (N, 394) where N is the number of windows, ready for the augmentation loop.

Stage 3 — Augmentation Loop

This is the stage that does the most work computationally. For each real window, ADACL generates K augmented variants using the toolkit from Lesson 32 — parametric drift simulation, multi-qubit anomaly injection, environmental-context conditioning, adversarial augmentation. K is typically 100–1000 depending on the operating budget and label scarcity.

Augmentation outputs three things per synthetic example:

1. The augmented feature vector (a perturbation of the original 394 features).
2. The augmentation operator and parameters used (metadata, kept for explainability).
3. The continuous deviation label (computed in stage 4 from the augmentation parameters).

By the end of stage 3, the effective training dataset is 100×–1000× larger than the raw labelled set.

Stage 4 — Continuous Labelling

Each example — real or augmented — gets a continuous label computed by the physics-informed deviation function (Lesson 33). The function combines the DeCoN-PINN prediction error, the magnitude of the applied augmentation, and any physics-constraint residuals into a scalar in [0, 1].

For real labelled positives, the operator tag is mapped to the continuous label via the calibration table held out for that purpose (see stage 6). For real labelled negatives, the label is 0 by definition.

The result is a paired training set (features, label) where label is a meaningful continuous deviation magnitude, not a class.

Stage 5 — CNN Regression Training

The CNN regressor (Lesson 33) is trained on the paired dataset using the hybrid MSE + hinge loss. Specifics:

• Weights are initialised with Xavier (He for ReLU paths) — Lesson 36.
• Optimiser: AdamW with weight decay; LR schedule warms up over 1k steps then cosine-decays over the rest of training.
• Batch size and total step count are set by the size of the augmented dataset; typical runs are tens of thousands of steps.
• Validation: hold out 10 % of the real labelled positives + a stratified slice of augmentations as a validation set. Monitor MSE on validation each epoch; early-stop if it plateaus.

The output of stage 5 is the trained CNN weights — but those alone are not yet a deployable model.

Stage 6 — Calibration & Validation

The trained CNN emits scores, but its raw scores may not be perfectly calibrated to operator judgment. The calibration stage tunes a monotone post-hoc transformation (typically isotonic regression or Platt scaling) that maps raw CNN outputs to operator-aligned scores in [0, 1].

Calibration input: a held-out set of operator-validated examples (real positives + false-positive review data + confirmed negatives). The calibration step finds the transformation that minimises the difference between the model’s score and the operator’s judgment on this set.

Validation then runs the calibrated pipeline on a separate held-out slice and reports:

• Calibration error: how close are model scores to operator-aligned scores on average?
• Ranking quality: across alerts above various thresholds, what fraction were operator-confirmed?
• Coverage in low-data regions: where labels are scarce, is the model still calibrated or has the augmentation distribution dominated?
• Robustness to held-out drift modes: does the model generalise to drift trajectories it was never augmented with?

A model that fails any of these is rejected — it does not ship.

Stage 7 — Deployment Handoff

The package that goes to production:

1. CNN model weights (typically a few MB).
2. Feature-engineering recipe — the exact list of feature names, their compute formulas, and their version (so the deployed system computes features in the same order the model expects).
3. Calibration table — the post-hoc score transformation.
4. Augmentation metadata bundle — for explainability, the deployed serving system needs to know what augmentation classes the model was trained on.
5. Validation report — the numbers from stage 6, captured at training time, attached to the model for audit.

Together this is the deployable artifact. Module 7 covers what happens after deployment — adaptive baselines, explainability, the feedback loop that closes the development cycle.

The Feedback Loop

A pipeline that runs once and then sits frozen is not useful. ADACL closes the loop:

• Operator feedback on each production alert (true positive, false positive, marginal) is captured.
• Periodically, the captured feedback is re-incorporated into the labelled-positive corpus.
• The pipeline is re-run from stage 1, producing a new model trained on the broader corpus.
• The new model is calibrated against the latest operator-validated set, validated against the held-out slice, and (if it passes) shipped.

The frequency of retraining is operationally tuned: typically weekly for active systems, monthly for stable ones. The retraining is automated; the validation gate is the only manual step.

Common Pitfalls

Three failure modes seen in early ADACL deployments — worth flagging here so they can be designed around:

• Augmentation distribution drift: if the augmentation parameters are not updated as the real system evolves, the model learns a curriculum that no longer matches reality. Mitigation: monitor the distribution of real labelled positives over time and update augmentation parameters to match.
• Calibration silently degrading: post-hoc calibration assumes the validation set is representative. If the operating regime shifts substantially, the calibration drifts. Mitigation: track calibration error on rolling-window operator feedback and trigger retraining when it crosses a threshold.
• Feedback loop poisoning: if operators systematically dismiss true positives during alarm fatigue, those examples enter the corpus as false-positive training data and corrupt the next model. Mitigation: weight feedback by operator confidence + cross-validate against the held-out gold set.

These pitfalls are not unique to ADACL — every ML system in production faces some version of each. ADACL’s framework just makes them tractable because each stage is named and inspectable.

Key Takeaways

• Understanding the fundamental concepts: The ADACL training pipeline has seven explicit stages — ingestion, feature engineering, augmentation, labelling, CNN training, calibration, deployment. Each stage has a defined input/output contract; problems can be localised to a stage rather than chased across the whole system.
• Practical applications in quantum computing: The augmentation loop and the continuous-labelling stage are the parts that make ADACL practical in the small-label quantum regime. Without them the pipeline collapses to a generic classifier trained on a hundred examples — which is to say, doesn’t work.
• Connection to the broader SNAP ADS framework: Module 6 ends here, with the full pipeline laid out. Module 7 zooms into the technical machinery (architecture, initialization, features), Module 8 covers performance and validation in production, Module 9 covers deployment across application domains.

What’s Next?

Module 7 opens with ADACL’s neural-network architecture in detail — the specific CNN layers, the choice of activation functions, the regularisation strategy, and the way the input feature tensor is shaped. After that we move to Xavier initialization (Lesson 36) and the 394 physics-informed features (Lesson 37) we’ve been referring to throughout this module.

---

Ready to continue? Use the navigation buttons below to move to the next lesson or return to the module overview.