The sense that touches back

Aug 03, 2026 · #robotics #tactile-sensing #manipulation #perception

Machine touch is not a resolution problem. A tactile sensor must deform to register anything, which makes it part of the contact it reports, and the useful signal lives in a control loop far faster than perception. Why cameras under rubber won, and what that design costs.

An essay card showing two gripper fingers with deformable gel pads pressing on an object

A two finger gripper closes on a disposable coffee cup. The vision stack has already done the part that looks hard: it localised the cup, estimated its pose, chose an approach direction and picked a grasp that a person would also have picked. Then the fingers close, and the cup either crumples or squirts sideways and lands on the table.

Both failures happen in the last two centimetres, after perception has finished and while the only information that matters in the world is arriving through a patch of contact a few millimetres across.

This is the part of manipulation that has not moved at the pace of the rest. Detection, pose estimation and motion planning have been largely absorbed by learned models. Contact has not. The standard diagnosis is that tactile sensors remain primitive and that better ones will close the gap. That diagnosis is wrong in a way worth unpacking, because high resolution tactile sensors have existed for more than a decade. The unsolved part is that touch, alone among the senses a robot has, cannot be sampled without being disturbed, and that its useful signal lives in a loop far faster than anything we would normally call perception.

A measurement that changes what it measures

Every other robotic sense is remote. A camera collects photons the scene was emitting anyway. A lidar’s own photons perturb nothing they bounce off. A microphone samples pressure waves that would have propagated regardless. In all of these the instrument can be arbitrarily stiff, arbitrarily well protected, and arbitrarily far from its subject.

Touch has none of that. The sensor is the interface. To register a force it has to deform, and its deformation is part of the mechanics of the very contact it is trying to report. Make the surface soft enough to register a light brush and you have also made it soft enough to change how the object rolls under the finger, how wide the friction cone opens, and where the pressure concentrates. There is no arrangement in which the instrument stands outside the experiment.

That single fact produces most of the field’s constraints, and they pull against one another.

Compliance fights bandwidth. A soft elastomer is viscoelastic, so it creeps under sustained load, shows hysteresis on release, and behaves as a mechanical low pass filter whose corner frequency sits in the tens of hertz. Stiffen it and the response gets fast and faithful, at the price of going blind to gentle contact and concentrating stress on whatever you are holding.

Compliance also fights durability. An eye can live behind glass. A fingertip is abraded every time it does its job. Tactile skins usually fail by wear, tearing and delamination rather than by electrical drift, which is a failure mode that no amount of calibration recovers.

And coverage fights wiring. Discrete sensing points need routing, and a genuinely covered humanoid implies thousands of them, each connection surviving repeated stretch at a joint. Skin is not a sensor. It is a manufacturing problem in the shape of a sensor.

The fingertip is not a high resolution camera

The human comparison gets invoked constantly and usually backwards. Human glabrous skin does carry a dense receptor population, but the reason people can hold an egg is not spatial acuity. It is the structure of the loop.

The classic microneurography work on precision grip, and the line of studies that followed it, describe a system whose key trick is anticipation plus fast correction. Fast adapting type I afferents respond to the small localised slips that precede full macroscopic sliding, and the motor system reacts on a timescale that is genuinely reflexive: measurements in this literature put the sensory transit to the motor system at roughly fifty milliseconds, a complete corrective response near seventy, and observable grip force adaptation to an unexpected change in surface friction around one hundred milliseconds after contact.

The important consequence is what the system never does. It never measures the coefficient of friction, because friction is not observable without sliding. Instead it grips with a safety margin above the minimum force it predicts will be needed, based on a memory of how this object behaved last time, and it treats the first micro slips as an error signal saying the prediction was too optimistic. Touch here is not building a picture of the world. It is supplying the error term of a controller.

That reframes the engineering target. Detecting the onset of an event within milliseconds is worth more than resolving its shape in exquisite detail a tenth of a second later.

Cameras under rubber

The design that came to dominate high resolution tactile sensing sidesteps the wiring problem entirely by turning touch into vision. Put a small camera inside a hollow fingertip. Fill the tip with clear elastomer coated on the outside with an opaque reflective membrane. Illuminate the membrane from the edges with coloured light arriving at different angles. When something presses the surface, the membrane’s local slope changes, each colour channel brightens or darkens according to that slope, and photometric stereo reconstructs a height map of the contact patch. Print markers into the gel and their displacement gives you shear as well.

The trade is excellent on its face. One cable and one commodity image sensor replace a dense array of taxels, and the effective spatial resolution is high enough to read the raised lettering on a screw head or to see the individual threads of a fabric. Geometry, the quantity that grasp planning most wants and that vision loses at the moment of occlusion, comes out directly.

The costs are structural rather than incidental. A camera needs focal distance, so the fingertip becomes a bulb, which is exactly the wrong shape for reaching into a cluttered bin. The gel is a consumable that clouds and tears. And the latency budget is unforgiving: a sixty hertz camera has already spent sixteen milliseconds accumulating a frame before a network has seen a single pixel, and the network then costs its own few milliseconds, and the controller acts on the result some time after that.

Neither constraint is about resolution, and both survive any improvement in sensor materials.

That latency argument explains a persistent minority tradition: event driven tactile front ends. Rather than frames, a per pixel change detector emits asynchronous events with microsecond timestamps, so the sensor natively reports onsets, which is precisely the quantity slip detection needs. Several 2026 designs push in that direction, including flickering illumination schemes for photometric stereo on event cameras and miniaturised spiking visuotactile sensors. The literature is tracked in reasonable order by the Awesome-Touch bibliography, which is a better map of the field than any single review.

The other family, and why it refuses to die

Alongside optical tactile sensing sits the older electrical tradition: capacitive, piezoresistive and triboelectric arrays laid on a flexible substrate. Its recent progress is not mainly about sensitivity, which was adequate years ago, but about manufacture. Work published through 2026 emphasises building arrays directly onto curved three dimensional substrates rather than fabricating a flat sheet and wrapping it, with one group reporting several hundred sensing elements integrated on a sub metre film using printing plus laser direct writing, and others pursuing scalable in situ fabrication of multimodal skins. Printed triboelectric skins now claim conformal coverage of a whole robotic hand, fingertips through dorsum.

A more radical variant abandons the array altogether. Instead of routing to every sensing point, place electrodes only at the boundary of a continuous conductive elastomer and reconstruct the interior contact field by solving an inverse problem, a strategy demonstrated with combined electrical impedance and acoustic tomography in a biomimetic robot skin. The wiring collapses to the perimeter. The catch is intrinsic: the reconstruction is ill posed, so accuracy degrades away from the electrodes and degrades further when several contacts occur at once, which is the normal condition for a limb moving through clutter.

This family persists for a reason that has nothing to do with resolution. A humanoid needs to know that its forearm brushed a shelf, that its shin met a pallet, that a person leaned on its back. Thin, tough, cheap, whole body coverage answers a different question than reading a screw head, and both questions are real.

Where this stands in 2026

The centre of gravity has moved from sensor design to representation, and for a specific reason: touch has nothing resembling a common corpus. A vision-based tactile image is a function of that particular gel’s thickness, that particular set of LED angles, that particular membrane. Two sensors of the same lineage with different gel depth produce incomparable readings from an identical contact. Every lab has therefore been training on data that transfers to nobody.

Consequently the visible work of the past two years is about cross sensor representations and about folding touch into the vision language action stack that now dominates manipulation. Recent efforts include semantically aligned vision, tactile, language and action models, a shift from aligning touch with vision toward aligning it with force, and tactile foundation models that pair a slow high level policy with a fast tactile refinement loop underneath. That last architecture is worth noticing, because it is the same division of labour the spinal cord uses: low rate intention above, high rate contact regulation below.

What breaks

Several things, and they are not minor.

Benchmark numbers are frequently not measuring manipulation. Perfect classification accuracy over a closed set of pressed textures is a pattern recognition result on a fixed alphabet. It says little about whether a hand can hold an unfamiliar object it is allowed to touch only once.

Durability is under reported. Papers typically characterise sensitivity and hysteresis over a few thousand cycles. An industrial gripper works in the millions, and the failure modes that matter, membrane wear and delamination at a stretched interface, do not appear until well past the tested range.

Simulation is at its weakest exactly where the value is. Contact is the hardest thing to simulate faithfully: friction is not a constant, elastomer contact is history dependent, and stick to slip transitions depend on microscale surface state. Tactile simulators are reasonable at rendering indentation geometry and considerably weaker on shear, adhesion and slip onset, which is the signal a controller actually consumes. A policy trained mostly in simulation inherits that gap.

Most of the foundation model results cited above are preprints, generally produced by the groups that built the sensor, evaluated on their own hardware. That is a normal stage for a field, and it is not evidence of general capability.

The deepest problem is that touch data is self generated. Images can be scraped by the billion because the world produced them without any robot’s help. Contact data exists only if a robot touches something, which means a clumsy policy collects a clumsy dataset, and the scaling recipe that worked for language and vision has no equivalent starting corpus. Automated acquisition rigs are being built to attack this, and they remain small next to the web.

None of which means the coffee cup is unsolvable. It means the failure is not waiting on a better pixel. It is waiting on a control loop that can register the first millisecond of a slip, on a skin that survives a year of doing so, and on a way to accumulate contact experience that does not require every robot to learn it alone.