Why robot hands are harder than robot legs

Jul 28, 2026 · #robotics #manipulation #control #tactile-sensing

A humanoid that walks is a solved-enough control problem. A humanoid that takes a key out of a pocket is not. The gap is not precision or motor count: it is that contact is a discontinuity, and everything downstream, from simulation to grasp planning to touch sensing, inherits that fact.

Essay cover: Why robot hands are harder than robot legs, with two friction cones, one containing a force vector and one not

Humanoid locomotion has quietly become an engineering problem rather than a research problem. Balance is regulation under constraint, the models are good, and reinforcement learning in simulation transfers to hardware well enough that the videos are boring now. Manipulation has not followed. The same robot that jogs across a car park cannot reliably take a key out of a coat pocket.

Hans Moravec put the general shape of this in 1988: the capabilities evolution spent hundreds of millions of years polishing, perception and manipulation, are the expensive ones to reproduce, while abstract reasoning, bolted on late, turns out to be comparatively cheap to program. The hand version of the paradox is sharper than the usual telling. A machine can write a mathematical proof and still fail to turn a key.

The reason fits in one sentence, and the rest of this essay unpacks it: walking is a problem of continuous regulation, while manipulating is a problem of contact, and contact is mathematically a discontinuity.

Grasping is not dexterity

The first distinction almost everyone skips. Two very different things hide behind the word “manipulate”.

Grasping means securing the object to the hand and then moving it with the arm. The object does not move relative to the hand. This has been essentially solved in industry since the mid-2010s: a two-finger gripper plus a vision network is enough to empty a bin of unknown parts.

In-hand manipulation means changing the pose of the object within the hand: rotating a pen between the fingers, flipping a key to present the right edge, re-seating a grip without putting the object down. This is still open, and it is where the boundary between a gripper and a hand actually runs.

Human dexterity is almost entirely the second category, and it demands something counter-intuitive: letting go. To rotate an object in the hand, some fingers must release while others hold, which means deliberately passing through states where the grasp is not stable. A controller that maximises grasp stability at every instant will never rotate anything. It will hold the object beautifully, forever.

Contact is where the physics stops being smooth

As a finger approaches an object, the contact force is zero, zero, zero, and then suddenly something. There is no gentle transition. In dynamics terms the system is non-smooth: it changes mode. And at every contact point, three modes coexist.

Separated: no contact, no force.
Stick: contact without sliding. The tangential force is whatever it needs to be.
Slip: contact with sliding. The tangential force saturates and opposes motion.

With n contact points, the number of mode combinations is on the order of 3ⁿ. A five-fingered hand with three contact patches per finger gives 3¹⁵, roughly fourteen million discrete configurations, and the controller has to pick the right one every millisecond. That is the real combinatorial explosion of manipulation. Not the number of motors: the number of contact regimes.

The law governing the switch between stick and slip is Coulomb friction: the tangential force magnitude cannot exceed the friction coefficient times the normal force. Do not memorise the inequality, keep the cone. Picture a cone planted on the surface, apex at the contact point, opening outward along the normal. If the force vector you apply stays inside the cone, the contact holds. If it leaves the cone, it slips. A stickier surface means a wider cone, which means you are allowed to push more sideways. On ice the cone is a thread, and the smallest lateral component sends everything flying.

The whole stability of a grasp reduces to a geometric question: do the force vectors stay inside their cones? It is also why you squeeze a wet glass harder. You are not changing the cone, you are lengthening the vector so that the available lateral component grows with it.

Grasp theory, stripped of its notation, is one question repeated: does there exist a set of forces, each inside its own cone, whose sum cancels the object’s weight and torques?

Why the simulator lies

The method that unlocked locomotion is massive training in simulation followed by transfer to hardware. It works for walking because the dynamics of a body in flight or in firm support are well modelled. For manipulation it falls apart, for a precise reason: the simulator does not simulate contact, it approximates it, and every approximation has a personality.

Penalty-based solvers let bodies interpenetrate slightly and then push them apart like a very stiff spring. Fast and simple, but the stiffness you chose becomes a ghost physical parameter with no counterpart in reality. Constraint-based solvers instead resolve a complementarity problem at every timestep, either there is a gap and no force, or a force and no gap. More faithful, far more expensive, and numerically fragile once contacts multiply. And real friction is not a constant coefficient at all: it depends on velocity, on the history of the contact, on humidity, on how worn the fingertip is.

So a policy trained in simulation learns to exploit the simulator’s bugs. It discovers that pushing at a certain angle makes objects “stick” in a way real physics does not allow, and then fails inexplicably on hardware. The literature calls this the sim-to-real gap, but for manipulation the name is too vague. It is a contact gap. The productive response has been to stop chasing fidelity and instead make the policy indifferent to the contact model: aggressively randomise friction and stiffness during training, or learn compliance parameters directly rather than trajectories.

Control a force, not a position

An industrial arm is servoed on position. Give it a setpoint, it reaches it, and if an obstacle gets in the way it pushes until something yields. That is precisely the wrong behaviour around a fragile object. The answer is impedance control, and the equation matters less than the image behind it.

You do not connect the end effector to its setpoint with a rigid bar. You connect it with a virtual spring and damper. The robot stops trying to “be at this position” and starts trying to “behave as if it were attached to this position by a spring of stiffness K”. If the object resists, the spring stretches and the force stays bounded, automatically, with no force sensor and no decision logic. Low K gives a soft, compliant hand; high K gives a precise, brutal one.

Two philosophies of command. The second needs no force sensor: the force limit emerges from the imposed behaviour rather than from a measurement followed by a decision. Safety by construction instead of safety by surveillance.

Dexterity, in this framing, is not aiming better. It is modulating stiffness in real time across the phases of a gesture. Humans do it without noticing: stiff while approaching the lock, soft at the instant of contact.

Touch, reframed as a camera problem

To modulate stiffness at the right moment you first need to know you are touching. The approach that won here is elegantly indirect. Rather than manufacture thousands of reliable, individually wired pressure cells, vision-based tactile sensing puts a block of transparent elastomer at the fingertip, coats its outer face with an opaque reflective membrane, lights that membrane from the inside with LEDs of different colours at different angles, and films the whole thing with a tiny camera in the finger.

When the finger touches something, the gel deforms and takes the microgeometry of the surface. The coating, lit obliquely by distinct colours, returns a gradient that encodes the local slope at every point. A little photometry reconstructs the 3D topography of the contact, at a resolution better than a human fingertip: the weave of a fabric, the relief on a coin, the onset of a slip.

Touch turned into a vision problem. The trick is not the sensor, it is the change of discipline: instead of inventing a difficult new sense, you convert touch into images and inherit thirty years of computer vision, GPUs and convolutional networks.

The redirection has a cost. A human hand carries roughly 17,000 mechanoreceptors across four families that do not measure the same thing: two slowly adapting (form and static detail, skin stretch) and two rapidly adapting (incipient slip, and vibration up to several hundred hertz). An optical sensor delivers a very rich image, but in one modality, at camera frame rate, with processing latency. What actually stops a can from escaping your grip is not a beautiful contact image. It is a reflex driven by the fast-adapting units, which detect the micro-vibrations of an incipient slip and raise grip force within tens of milliseconds, before you are aware of anything.

Which is the general lesson. A hand is not an effector with sensors attached. It is a tight sensorimotor loop in which measurement and action are so interleaved that separating them destroys the function. A robot that perceives perfectly but reacts 200 ms late catches nothing.

Where this stands in 2026

Faced with a wrong model and an explosive combinatorics, robotics has made the same turn vision made in 2012 and language in 2020: drop the explicit model, learn the policy from demonstrations. Vision-language-action models take images and a natural-language instruction and emit joint commands directly at around 50 Hz. Ask for “put the blue cup away” and the model produces motor torques. No contact equation appears anywhere.

That turn hits a wall language never had: the data does not exist. Web text was already there, free, in absurd quantity. Manipulation trajectories have to be manufactured one at a time, by teleoperating a real robot with a real human in front of a real object. This is the true rate limiter of the field, and it explains most of the current research activity.

Three developments this year mark where the frontier sits.

Touch stopped being a fingertip feature. At CES 2026 several manufacturers made tactile skin the robotics announcement of the show. Ensuring Technology demonstrated a dexterous hand covered end to end in its Tacta sensor, with a claimed order of magnitude around two thousand multi-dimensional sensels on a single hand spanning fingertips, phalanges and palm. Sharpa, in Singapore, took an innovation award for a hand whose every fingertip pairs a miniature camera with more than a thousand tactile pixels. A fair caveat: these are trade-show claims, the density and sampling-rate figures are vendor numbers rather than peer-reviewed ones, and none of it demonstrates a useful task completed. Treat it as industrial intent, not as a result. Still, the move from “the fingertip senses” to “the whole hand senses” is a change of kind rather than degree, because enveloping grasps and mid-gesture regrasping are exactly what in-hand manipulation requires.

Benchmarks started asking the right question. DexVerse, posted to arXiv in July 2026, proposes a modular, multi-task and, crucially, multi-embodiment benchmark for dexterous manipulation: the same task must be solved by hands of different geometries. That attacks the fact that a manipulation policy is currently a prisoner of the hardware it was trained on. In parallel, work on retargeting human hand poses extracted from ordinary video onto robot kinematics, then compositing the robot arm back into the frame, reports real-robot success-rate gains in the range of a few to roughly twenty points depending on the task. It attacks the data wall through the one abundant reservoir that exists: video of human hands doing things.

Compliance became a learned variable. A 2026 survey in The International Journal of Robotics Research reviews imitation learning for contact-rich tasks, and the trend it reports is clear: policies increasingly learn impedance parameters jointly with trajectories, and simulator contact parameters are calibrated rather than guessed. What is worth learning from a gesture is not the path. It is the compliance along the path.

The loop, not the limb

One last observation, because it reframes the problem. Teleoperation is the dumbest technique in the field and still one of the most effective. A human wears a glove, a robotic hand a metre or a thousand kilometres away copies the fingers, and if visual and haptic feedback arrive fast enough, the operator stops “driving” and starts touching. The remote hand is experienced as their own.

The brain will annex an external object into its body schema on surprisingly thin evidence: temporal coherence between what it commands and what it perceives is close to sufficient. Which suggests that what separates a person from a distant body is not a deep question about substance. It is haptic delay. A body is less a container than an agreement about latency, and manipulation is where that agreement is hardest to keep, because contact is the one signal that punishes lateness immediately.