From 15 Minutes to 25 Seconds: Reinforcement Learning Accelerates Two-Qubit Gate Calibration 

Before a quantum algorithm can run, the quantum processor unit (QPU) must first be brought into a narrow, fine-tuned operating regime with reliable single- and two-qubit gates.  

Manually bringing up one high-fidelity two-qubit gate can take around 15 minutes. That is already expensive for a single gate. It becomes untenable for processors with hundreds or thousands of interacting qubits, where long-running workloads need calibration routines that are faster, more autonomous, and able to optimize many drifting parameters at once. 

Quantum Machines and the Thematic Center for Quantum Computing at Research Center for Critical Issues, Academia Sinica showed that quantum-classical hybrid control offers a solution to this. 

A two-qubit gate that normally takes about 15 minutes to bring up by hand can be calibrated in about 25 seconds when the control loop is handed to a reinforcement-learning agent. By connecting an OPX1000 controller directly to a classical GPU accelerator through OPNIC, the team enabled the agent to tune the gate continuously using real-time hardware feedback. 

That is the result Quantum Machines and Academia Sinica demonstrated by connecting an OPX1000 controller to a classical GPU accelerator through OPNIC, letting the agent tune the gate directly from hardware feedback. Taking a step further, the agent also optimized every parameter of a five-qubit circuit at once to prepare a GHZ state. The parameters still drift. What changes is how quickly they can be recovered. The difference is that getting them back takes seconds, not a fresh manual calibration.  

This work points toward the kind of simultaneous, multi-qubit, continuous calibration routines needed to operate larger quantum processors. 

Why is gate calibration hard? 

Calibrating a QPU means finding the narrow operating regime in which its control parameters produce reliable single- and two-qubit gates. In practice, that makes calibration a high-dimensional experimental search across pulse amplitudes, frequencies, phases, timings, and coupler settings. 

The challenge is that this operating point does not stay fixed. Each QPU carries small, device-specific imperfections from fabrication, and its environment drifts over time. Calibration is therefore not only a multi-parameter optimization problem but also a moving-target optimization problem, where the optimal control settings must be found quickly and updated repeatedly.  

That requirement is getting sharper as the outlook for utility-scale quantum computing changes. Recent progress in codes, architectures, hardware, and compilations have shown that fault-tolerant machines may need orders of magnitudes fewer qubits than earlier surface-code estimates implied, but they still have to run millions to billions of gates continuously, with minimal downtime and tightly integrated classical compute for decoding, feedback, control, and calibration. In that context, a manual calibration loop becomes the bottleneck. 

Manually bringing up a single high-fidelity two-qubit gate can take on the order of 15 minutes. For processors with hundreds or thousands of interacting qubits, and for workloads that must remain stable over long runtimes, calibration must become faster, more autonomous, and capable of coordinating many parameters at once. Within bring up, what has to be retuned depends on the gate, and the two-qubit gates are the expensive ones. The controlled-Z in particular is hardware dependent, so how you calibrate it, and how often, is set by the architecture itself. 

Academia Sinica’s tunable qubits with tunable couplers offer the testbed to see this in action. Each qubit’s frequency, single and two qubit gates are controlled via the OPX1000 MW FEM drives. In principle this tunability is what can make the architecture run fast but it is also what drifts, and what must be retuned. 

Each qubit’s frequency is set by a flux line, and the coupling between neighbors is set by a separate flux line on the tunable coupler. Raising or lowering the coupler’s frequency with flux changes how strongly the two qubits interact, which is what turns the CZ on and off. 

How does a CZ gate work? 

The gate uses the fact that the computational state |11⟩ sits close in energy to the non-computational state |02⟩, where both excitations occupy one of the qubits. When the qubits interact via the coupler, these two levels repel each other, shifting the energy of |11⟩ state. 

Holding the |11⟩ at that shifted energy for the right duration accumulates a conditional phase of pi needed for the CZ gate.  But the same flux excursion also changes the individual qubit frequencies during the gate. As a result, each qubit accumulates an additional single-qubit phase. If those phases are not corrected, they appear as coherent errors. 

This is why the full operation of applying a CZ gate requires applying compensation pulses to the single qubits to correct the coherent error. The gate is run adiabatically, ramping the coupler flux slowly enough that population stays in |11⟩ and does not leak into |02⟩. In this simplified view, a single CZ comes down to three numbers: the coupler flux pulse that sets the conditional phase, and two single-qubit phase corrections, φ_qc on the control qubit and φ_qt on the target, that cancel the extra phase each qubit picks up from the flux excursion. 

In practice, a typical bring-up runs as a sequence: the coupler-versus-qubit flux sweep, then CZ phase calibration, then error amplification of that phase, then leakage-error amplification, then single-qubit phase compensation, then error amplification of that compensation, and finally the confusion matrix of the parameters. 

Each step is hard on its own, a multidimensional parameter optimization that often means going back and forth between steps to adjust. It is harder still because you are calibrating against an environment that does not hold still. 

Done by hand, a single two-qubit gate takes about 15 minutes. The flux sweeps to find the operating point is the longest, requiring five minutes. CZ phase calibration adds three, the leakage check adds four, the single-qubit phase compensation adds three, and finally one minute for the confusion matrix.  

In superconducting circuits, the relevant parameters drift on timescales far shorter than the runtimes these algorithms demand. A 15-minute recalibration is not something you can afford to run mid-computation. Unless calibration can respond to the environment as fast as the environment changes, finding the optimized parameters in this search problem quickly, the gate quality and the computation degrade together. 

Most of that difficulty concentrates in the residual Z-phases, and their sensitivity to small parameter changes. Distortion and crosstalk matter too, but those are largely one-time calibrations. The Z-phases have to be tracked continuously as the drifts feed straight into computation quality. Fidelity can fall by 5 to 15 percent depending on how far the parameters have wandered. You see it most directly in the Bell-state fidelity after a CZ. What is needed is something that can keep the fidelity up by it enables rapid retuning, allowing us to bring the system state back up to date in a very short time. 

Reinforcement learning takes over the calibration loop

Academia Sinica fabricates their QPUs in-house, on the same kind of wafer-scale semiconductor process that Taiwan built its industry on. The QPUSs are meant to grow into a platform for industrial-scale algorithm development. Deployment at that scale requires control that is in situ, fast, and at high fidelity, continuously, while it computes.  

This means that the control system has to adapt and respond to the environment as it changes. Not one high fidelity bring up, but watch the system, react, and converge on the right parameters while the system is up. 

That is a closed loop, and closing it fast is a hardware problem.  

The measurement happens on the quantum chip, but the heavy computation, fitting, optimizing, deciding the next move, wants real compute. Standard control hardware cannot move data between the two fast enough to keep up with the drift it is chasing. 

Quantum Machines’ OPNIC changes that. It connects the OPX1000 controller directly to a classical accelerator, so high-compute work can run in the calibration loop in real time, with the controller feeding measurements and parameters back on microsecond timescales. The CZ bring up tested this. The autonomous calibration called for continuously learning about the environment in real time, from the environment itself. There is not enough data to train a conventional machine-learning model offline and deploy it. Instead, reinforcement learning fits the problem as it is built for autonomous agents that learn from their own interactions with an environment. 

The agent generates its own data by interacting with the QPU. That data is temporally correlated and never comes labeled with the correct answer. The agent reasons from what it observes to infer the right move and explores actively to make sure its data covers enough of the parameter space. 

The partnership between Academia Sinica and Quantum Machines leveraged an RL agent’s ability to outpace conventional calibration timelines.  

The collaboration used TD3, a publicly available reinforcement-learning agent, and placed it directly in the calibration loop through the OPX1000–DGX connection. In each iteration, the agent proposed a new set of control parameters, transferred them to the OPX1000 in roughly 2 µs, and the controller executed the experiment on the QPU. Each parameter setting was evaluated over 100 shots to suppress measurement noise. The OPX1000 then computed the confusion-matrix correction and reward, returned the averaged fidelity to the agent, and the agent updated its policy before sending the next parameter set. One full loop therefore included parameter proposal, hardware execution, measurement, reward computation, and policy update. 

For CZ calibration, the loop converged in about 1,000 iterations. For the five-qubit GHZ experiment, it used about 2,000 iterations. The CZ calibration was completed in 25 seconds and the full GHZ state optimization was completed in 2-3 minutes.  

For the CZ gate, the learned parameters matched, and in some cases exceeded, the reference calibration. The gain was not that reinforcement learning found an unreachable optimum. But it found the right operating point fast enough that calibration could become part of normal operation rather than an interruption to it. 

The GHZ result makes that result sharper. Starting from an unoptimized five-qubit circuit with about 4% GHZ fidelity, the agent raised the fidelity to about 77%, nearly a 20× improvement. The agent optimized the coupled circuit in one closed loop, instead of stepping through each gate and correcting it by hand. That is the conceptual step up: the optimizer is no longer only tuning a single gate, but a circuit whose parameters are coupled across time. In this way, OPNIC assisted hybrid quantum-classical control not only optimized gate parameters but also circuit parameters. 

The parameters still have to be relearned after drifting. But that is the point. Without the fast OPX1000–OPNIC loop, each drift event sends the experiment back to a roughly fifteen-minute calibration cycle. With OPNIC in the loop, the system can be retuned on the order of 25 seconds. The calibration burden does not disappear; it moves into the runtime control layer, where it can keep pace with the hardware. 

Is the future classical?   

Until recently, resource estimates for utility-scale quantum computing placed fault-tolerant computation firmly in the teraquop regime, requiring enormous qubit counts and operation volumes. This year’s advances in both hardware and algorithms have compressed that outlook considerably. 

Recent estimates from Google, Pinnacle, and Oratomic suggest that portions of this cost can be reduced to the tens-of-millions-of-gates regime, often by trading qubit count against runtime, decoding overhead, or architectural assumptions. 

The challenge is shifting from simply building larger quantum processors to operating them efficiently at scale. Achieving that requires a control system capable of keeping pace with an increasingly demanding compute stack. 

Classical accelerators working alongside the QPU are emerging as a core part of that stack. While QPUs and FPGA-based controllers execute quantum operations, classical accelerators can process measurement outcomes in real time, enabling adaptive calibration, error suppression, and fidelity optimization. As quantum systems grow and complexity, increasingly sophisticated reinforcement-learning and optimization algorithms will be needed to navigate expanding parameter spaces and maintain peak performance. 

Software stacks, like the one used in this work, built entirely in Python, make it straightforward to integrate advanced optimization frameworks such as ORBIT into the control loop. The result is a flexible architecture that can evolve alongside both quantum hardware and classical AI capabilities. 

Want to see what that looks like in practice? Reach out to schedule a demo!