Deploying ADACL in Real-Time Systems

Welcome to Lesson 46 of the SNAP ADS Learning Hub! Module 9 is the practical “we have to ship this thing” half of the course. Lessons 1-45 built up the framework, demonstrated its gains, and laid out the validation and reproducibility discipline that keeps the gains durable. This module starts the conversation about deploying the framework — the operational engineering that surrounds the model itself.

Lesson 40 covered production optimisation at the model level (quantisation, tiered feature compute, calibration freshness). This lesson goes a layer outward: the system ADACL gets deployed into. Latency budgets, throughput design, deployment topology, on-call rotation, and the operational disciplines that determine whether the deployment is still working a year from now.

A useful framing: deploying a research-grade ADACL into production is like installing a high-precision instrument in a factory. The instrument is built well, but if you mount it on a vibrating chassis next to a heat source and forget to maintain it, the precision evaporates. The instrument has not failed; the deployment context has. This lesson is about the chassis, the heat shielding, and the maintenance schedule.

What “Real-Time” Means for ADACL

“Real-time” is a vocabulary minefield. Three different meanings show up in the anomaly-detection literature:

• Hard real-time: deterministic latency guarantees, missed deadlines cause system failure. Avionics, medical pacemakers.
• Soft real-time: predictable latency, occasional misses are tolerable but degrade quality. Video streaming, online gaming.
• Streaming/near-real-time: continuous processing with sub-minute latency, no per-message hard deadline. Most monitoring systems.

ADACL is streaming/near-real-time. Latency from feature compute to score publication should be under 30 seconds end-to-end. Missing that target occasionally is fine. Missing it for hours is not. Hard real-time guarantees are neither necessary nor achievable with a deep model in the inference loop.

This framing matters because the system around ADACL — the message bus, the feature pipeline, the alerting layer — has to be matched to the same latency regime. Streaming/near-real-time systems use Kafka or NATS, not real-time-OS scheduling. The whole stack lines up around the same SLA.

The Latency Budget

For a deployment with a 30-second end-to-end SLA, the budget breaks down approximately as:

Stage	Budget
Sensor → feature pipeline ingest	5s
Feature computation (tier 1)	5s
Model inference	1s
Calibration + post-processing	1s
Alert dispatch	5s
Slack/dashboard/page propagation	10s
Total	27s

The numbers are illustrative; real budgets vary. The point is that the model itself is a small fraction of the total latency. Optimising the model from 1s to 0.5s buys 0.5s on a 30s budget — a 1.7 % improvement. Optimising the alert dispatch from 5s to 1s buys 4s — a 13 % improvement. The biggest latency wins are usually outside the model.

Throughput Design

A typical ADACL deployment scores one alert per qubit per minute. For a 100-qubit system that’s 100 inferences per minute, or under 2 per second — trivial for a single inference node. For a 1000-qubit system it’s 16 per second, still single-node territory. For a 10000-qubit system you’d start needing multi-node inference, but no current quantum hardware approaches that scale; this is a deployment-future concern, not a deployment-now one.

The throughput bottleneck in practice is usually not inference but feature computation. The expensive features (tier 3: state tomography, randomised benchmarking) take seconds per qubit. If the feature pipeline can’t keep up, the inference loop starves regardless of how fast the model runs. ADACL’s tiered scheduler (Lesson 40) is the architectural answer: cheap features always-on, expensive features triggered or periodic.

Deployment Topology

Three common topologies for real-time ADACL deployments:

Co-located (single node)

All components — feature pipeline, model inference, calibration, alert dispatch — run on a single node next to the quantum-hardware control plane. Best for: small deployments, regulatory environments requiring physical co-location, lowest latency.

Trade-off: single point of failure. Mitigated by keeping the model and feature recipes in version control and having a documented “spin up replacement node from cold” runbook.

Hub-and-spoke (small cluster)

Feature pipelines run on edge nodes co-located with the quantum hardware; model inference and aggregation run on a central node. Best for: multi-device deployments, where the feature pipeline’s compute is per-device but the inference can be batched.

Trade-off: more moving parts. Mitigated by infrastructure-as-code so the topology is reproducible.

Disaggregated (managed services)

Feature pipelines in containerised services, model inference in a managed ML platform, alert dispatch in a managed observability platform. Best for: large operational organisations where each layer has its own team.

Trade-off: latency budget gets eaten by service-to-service hops. Often the right choice operationally despite the latency cost.

The default for new ADACL deployments is co-located, switching to hub-and-spoke when multi-device coverage is needed.

The Alert Path

Once a score is computed, the path from “model emits 0.87” to “operator sees an actionable alert” has its own design choices:

1. Score is written to a time-series store (typically Prometheus or InfluxDB) for dashboard rendering.
2. Policy layer reads the score against the configured thresholds for the deployment. The policy layer is outside the model — adjustable without retraining.
3. Alert is generated when a threshold is crossed. The alert carries the score, the model version, the calibration version, the top contributing features (from the explanation layer), and the recommended action from the runbook.
4. Alert is dispatched to the configured channel — Slack for low-severity, pager for high-severity. Dashboard renders the score in real time regardless.
5. Operator decision is captured — confirm, dismiss, escalate, mitigate. The decision goes into the operator-feedback rolling window that drives weekly recalibration.

The alert path is where most production bugs live. Dashboards lagging the alert by 30 seconds, alert payloads missing the model version, policy thresholds set in two places that disagree — these are the operational defects that erode trust faster than model accuracy regressions do.

On-Call Rotation

ADACL needs an on-call. The rotation typically has two roles:

• ML on-call — owns model behaviour. Investigates calibration drift, validation regressions, gold-set degradation. Authorised to roll back models.
• Hardware/operations on-call — owns the underlying system being monitored. Triages ADACL alerts, validates true/false positives, mitigates real anomalies. Provides feedback that closes the calibration loop.

The two rotations need to talk to each other regularly. When operators report sustained false-positive surges, that’s a model-side ticket. When the ML team notices feature distributions drifting outside training range, that’s a hardware-side ticket. The interface between the two rotations is one of the most important parts of operational ADACL.

Deployment Maturity Stages

A typical deployment matures over months:

• Month 0 (shadow): ADACL runs in shadow mode, scores are emitted to a private dashboard, no alerts fire. Operators get to look at the scores and provide initial feedback. Calibration is bootstrapped.
• Month 1 (advisory): Low-severity alerts only, dashboard-visible but no pager. Operators triage at their own pace. Operator feedback drives first calibration refresh.
• Month 3 (active): Full alert path live including pages. Drift dashboards and the gold-set audit are in steady operation. Weekly recalibration cycle established.
• Month 6+ (mature): Quarterly reviews of overall framework metrics. Hybrid composition with traditional methods (Lesson 43) considered if the regime benefits. Cross-deployment learnings shared across the operational org.

Trying to skip from month 0 to month 3 directly is the most common deployment failure mode. The shadow and advisory periods are not theatre — they’re how operators build the model of how ADACL behaves, and how ADACL gets calibrated against the specific deployment’s operator base.

What Determines a Year-One Survival

A deployment that’s still working a year in usually has all of these:

• Tight latency budget enforced via dashboards, not just SLAs.
• Tiered feature scheduling (cheap always-on, expensive triggered).
• Single point of policy-threshold ownership.
• Calibration refresh on a weekly cycle.
• ML on-call and ops on-call rotations with explicit handoff procedures.
• Gold-set audit on a weekly cycle.
• Drift dashboards reviewed at least monthly.
• Runbook for every alert severity.

Deployments missing any of these tend to degrade silently — the score numbers look fine, but operator trust erodes, alerts get dismissed at higher rates, and eventually the deployment stops mattering. ADACL was designed to be operationally honest; the operational discipline above is what keeps it that way in practice.

Key Takeaways

• Understanding the fundamental concepts: Real-time ADACL deployment is streaming/near-real-time with a 30-second end-to-end latency budget. The model is a small fraction of that budget; most latency wins are in feature compute, alert dispatch, and the system around the model.
• Practical applications in quantum computing: Most quantum-hardware ADACL deployments start co-located (single node), move to hub-and-spoke as device counts grow, and reserve disaggregated topologies for large operational organisations. Maturity stages from shadow (month 0) through mature (month 6+) are real and skipping them is the common failure mode.
• Connection to the broader SNAP ADS framework: Module 9 opens here, with the engineering around the model. The next four lessons go deeper into specific deployment surfaces — quantum hardware monitoring (Lesson 47), real-time drift APIs (Lesson 48), cross-domain applications (Lesson 49), and future directions (Lesson 50).

What’s Next?

Lesson 47 focuses on the canonical deployment surface for ADACL — quantum hardware monitoring. We will see what’s specific to that domain in terms of feature engineering, integration with control planes, and the operational interface between ML on-call and hardware-physics on-call.

---

Ready to continue? Use the navigation buttons below to move to the next lesson or return to the module overview.