The delay budget: what a few milliseconds do to a borrowed body

In 2024 a surgeon sitting in Bordeaux performed a partial nephrectomy on a patient in Beijing, roughly 8264 kilometres away, with a reported round trip latency of 132 milliseconds. Every account of that operation leads with the distance. The distance is the least interesting part. The number that decided whether the procedure was possible at all was the 132.

Delay is the tax every teleoperated system pays, and unlike most engineering constraints it cannot be bought down past a hard floor. It is not a quality-of-service problem to be solved with a better router. It changes what the operator can feel, what the controller can safely render, and, in a twist that has become commercially important in the last two years, what the recorded data is worth afterwards.

Where the milliseconds actually live

Start with the floor, because the floor is physics and everything above it is engineering.

Light in silica travels at about 204,000 kilometres per second, roughly two thirds of its vacuum speed, because the fibre core has a refractive index near 1.47. That works out to about 5 microseconds per kilometre in each direction, or 1 millisecond for every 100 kilometres of round trip path. For a great circle distance of 8264 kilometres the round trip is 16,528 kilometres, and pure propagation costs about 81 milliseconds.

That is already most of the 132. Fibre routes are not great circles either, so the real geographic floor sits higher than the map suggests. Whatever is left, and in this case it is something on the order of fifty milliseconds, has to cover the entire rest of the chain: camera exposure and readout, video encoding, packetisation, every queue in every switch along the path, decoding, display refresh, the servo loop on the robot, and the sampling period of the master console.

Two things follow. The first is that video is usually the largest term anyone can actually control, and every encoder setting trades latency against bitrate, which is why low latency telesurgery links look nothing like a streaming service. The second, and the one that gets under-reported, is that jitter matters more than the mean. A controller tuned for an average delay is not tuned for a delay that moves. A link with 120 milliseconds of steady delay is a better link than one averaging 90 milliseconds with 60 of variance, and the second one will look better in the press release.

There is also a vocabulary problem worth naming. Published latency figures are frequently network round trip times measured between endpoints, not glass-to-glass end to end delay from the moment the tissue moves to the moment the surgeon’s display shows it. The two can differ by a factor approaching two. When comparing two systems, the first question is which number is being quoted.

Why force feedback is the hard half

Vision tolerates delay reasonably well because it is a one-way stream. You watch a picture that is old, you adapt, you compensate. Force feedback is a different animal, because it closes a mechanical loop through the network. The network stops being a pipe and becomes a component of the mechanism.

The clean way to see the failure is through energy. Call a system passive if it can store or dissipate energy but never create it. Your arm is passive in this sense, so is a well designed manipulator, so is a block of tissue or a table edge. Connect several passive elements and the assembly is stable, no matter how you arrange it. That is an unusually strong guarantee, and it is the foundation of essentially every bilateral teleoperation controller in service.

Delay destroys it. The force that pushes back against your hand is the answer to a motion you made some tens of milliseconds ago. If you have moved on since, and you usually have, the returning force is not opposing your current velocity, and over a cycle the channel can hand back more energy than it received. A delayed communication channel is an active element. The energy has to appear somewhere, and it appears as oscillation: the console buzzes, the tool chatters against the surface, the operator instinctively grips harder, which stiffens the loop and makes the oscillation worse.

The classical fix is to stop transmitting force and velocity and instead transmit wave variables, particular linear combinations of the two chosen so that the channel is provably passive for any delay whatsoever, even an unbounded one. The guarantee is real and the price is real: position drift accumulates between master and slave, and waves partially reflect at the ends of the line, which distorts what the operator feels into something that is stable but not truthful. Recent work continues to attack exactly those two artefacts, for instance by removing force terms from the forward wave and adding energy reservoirs that keep the channel passive under time-varying delay.

The alternative family, time domain passivity control, is more opportunistic. An observer continuously tallies the energy flowing through the channel, and whenever the tally shows that energy has been created, a variable damper dissipates precisely that surplus and no more. It intervenes only when needed, which preserves more of the feel, at the cost of a guarantee that is now conditional on the observer being right.

Underneath both sits a trade with no way around it. Transparency, meaning the operator feels the remote environment’s true impedance, and robust stability are in direct opposition once delay is nonzero. A system that renders a hard contact perfectly at 150 milliseconds of round trip delay is a system about to oscillate. Every architecture in the literature, including the widely used four channel structure that decouples the position and force loops, is a particular choice of where to sit on that trade.

The operator is also a control loop

The human in the chair is not a passive consumer of this delay. Motor control runs on prediction: when you issue a command, a forward model produces an estimate of the sensory consequences before any feedback arrives, and that estimate is what you actually experience for the first fraction of a second. Small delays are invisible because the prediction covers them. Once the delay exceeds what prediction can bridge, the mismatch surfaces as an error signal, and the sense of directly inhabiting the remote tool degrades into the sense of driving it.

The behavioural signature of that transition has been documented since the earliest space and undersea teleoperation work, and it is called move and wait. The operator stops closing the loop continuously and starts issuing a small motion, waiting for confirmation, then issuing the next. Control becomes open loop in short bursts, and task completion time grows roughly in proportion to the delay, not because the hands are slower but because the strategy changed.

There is now physiological evidence for the cost. A study using functional near infrared spectroscopy found that latency in teleoperation significantly increased functional connectivity within and between the prefrontal and motor cortices, which is the neural picture of an operator spending executive resources to compensate. The same work found that supplying real time haptic feedback while the visual channel remained delayed reduced average functional connectivity across the cortical networks measured, which suggests that a fast touch channel can partly buy back the attention that a slow visual channel consumes.

Where this stands in 2026

The clinical numbers have converged into a rough consensus band. Round trip latencies at or below 200 milliseconds are treated as ideal for preserving hand and eye coordination, around 300 milliseconds remains workable, 400 to 500 milliseconds is described as acceptable but tiring, and 600 to 700 milliseconds is restricted to simple and low risk procedures. Japan’s 2022 remote surgery guidelines are stricter still, specifying end to end latency at or below 100 milliseconds together with a closed network and redundancy, and a five year validation of the hinotori system over 5G was published against that framework.

Real procedures are landing inside the band. A cross border Asia to Middle East series reported an average round trip latency of 166.6 milliseconds over fibre with 5G as backup. Simultaneous remote proctoring between Orlando and Shanghai reported a median of 139 milliseconds, with a spread of 137 to 216 on the first day. Those spreads are the interesting column, not the medians.

Meanwhile the same technology has acquired a completely different customer. Teleoperation is now the primary way manipulation data gets collected for robot learning: operators drive robots through leader-follower arms, VR interfaces or exoskeletons, and every episode becomes a training demonstration. Sanctuary added tactile sensing to its humanoid explicitly to give operators a richer feel, on the argument that better feel produces better demonstrations. That the flagship robotics conference is running a 2026 workshop titled Beyond Teleoperation tells you both how central the method has become and how much the field would like to stop depending on it.

This is where the delay budget acquires a second meaning. A demonstration recorded under latency does not record the task. It records an operator compensating for latency while doing the task: the hesitations, the overshoot corrections, the move and wait rhythm. A policy trained on those trajectories learns the compensation as if it were part of the skill, and then executes it on a robot that has no delay to compensate for.

What does not hold up

The clinical results deserve a discount. These are small, carefully selected series, run by expert surgeons on dedicated fibre with backup links and rehearsed contingency plans. They demonstrate feasibility under near ideal conditions. They are not evidence about routine practice over commodity networks, and a 2026 review in Annals of Medicine and Surgery makes precisely that argument, questioning whether current threshold claims are over-optimistic and noting how thin the validation is for the predictive and adaptive control schemes usually invoked to absorb residual lag.

Predictive display and model mediated teleoperation deserve a sharper warning. They work by showing the operator a simulation of the remote environment rather than the delayed truth, which is excellent while the model is correct and dangerous in a specific way when it is not: the display stays smooth and confident while diverging from reality, so the failure mode is silent.

Even haptics, the most promising mitigation, does not scale the way its advocates imply. A study of high latency teleoperation for space applications found that in real time conditions haptic feedback improved success rate, accuracy, contact force, velocity and operator trust, but at higher latencies only the contact force and velocity benefits survived. The accuracy and trust improvements were lost or reversed. That last word is the one to sit with. Adding force feedback to a badly delayed link can make an operator more confident about a signal that is stale, which is worse than giving them nothing and letting them know it.

None of this is a reason to stop building these systems. It is a reason to treat the latency figure as the specification it actually is. A remote body is a body you rent, and the delay budget is the lease.

Further reading

• Brain functional connectivity under teleoperation latency: an fNIRS study
• Impact of haptic feedback in high latency teleoperation for space applications
• Latency thresholds in telesurgery: are current clinical claims over-optimistic?
• Telesurgery: current status and strategies for latency reduction
• Three channel bilateral teleoperation with communication delay using wave variable compensators