The Geometry of Meaning: how a machine files words in space

Here is a deceptively hard question: how do you put meaning into a box? Not a word as a string of letters, but its sense: the fact that “surgeon” sits near “doctor”, miles from “beetroot”, and that between the two there is a smooth landscape of relatedness. How does a machine hold that? The answer, as simple as it is powerful, is one word: geometry. You turn every idea into a point in a space and let distance do the work. Those points are called embeddings.

First, a smaller trick: turning a person into five numbers

Forget words for a moment. Imagine summarising a person by a personality test, say the Big Five traits. You get five scores from 0 to 100: extraversion, openness, and so on. Line those five numbers up and you have a vector: an arrow in a five-dimensional space. Here is the move: to find who resembles you, no one needs to understand you. They compare your arrow to everyone else’s. The person whose arrow points the same way is your nearest match. We have quietly swapped “understand a person” for “measure the angle between two arrows.” That is exactly what we will do with words.

“You shall know a word by the company it keeps”

The line belongs to the linguist J.R. Firth (1957), and it is the whole trick. A word has no meaning on its own; its meaning is the set of contexts it appears in. “Cat” and “dog” haunt the same sentences (purrs, barks, bowl, vet), so their meanings are neighbours. In 2013 a Google team (Mikolov) turned this into a famous algorithm, word2vec: slide a small window across billions of sentences and, at each position, ask a tiny network to guess the neighbouring words. To guess well, the network is forced to give words that share company similar vectors. Nobody defines “royalty” or “feline”; those axes emerge on their own, out of sheer proximity. Hold that image: we inject no dictionary: we let the map draw itself from who-sits-next-to-whom.

Similarity is an angle, not a length

Once words are arrows, “how alike are these two?” becomes a question of geometry: look at the angle between their arrows. A tight angle (they point the same way) means very close meaning; a wide angle means strangers. This is the well-known cosine similarity. And the useful part is what you can safely ignore: the length of an arrow. Repeat a word ten times and its arrow gets longer, but it points the same way: same meaning. Direction is the meaning; length is only volume. Whenever you hear “cosine similarity”, just picture two arrows from a shared origin and ask how wide the angle is.

Doing arithmetic with meaning

Here is the result that stunned everyone in 2013. Take the arrow for “king”, subtract “man”, add “woman”. Where do you land? Right next to “queen”. You can compute with meaning: king − man + woman ≈ queen. Why? Because certain directions of the space have captured entire concepts. There is a “gender” direction (the step from man to woman) and a “royalty” direction; moving along one changes a single facet of meaning while leaving the rest intact.

Don’t oversell it: it isn’t perfect, and “queen” is only the nearest neighbour of the computed point, not a magic equality. But the idea is vertiginous: abstract relationships (gender, a country’s capital, the plural of a noun) have turned into regular geometric journeys. Meaning has a shape.

From three dimensions to three thousand

Our drawings cheat: they are 2D. Real embeddings live in spaces of 384, 768, 1,536, even 3,072 dimensions. Impossible to picture, and it doesn’t matter. Nobody “sees” in 768 dimensions, not even the researchers. The one rule you need (close = similar, tight angle = same meaning) holds no matter how many dimensions there are. Your little 2D intuition, the clusters and the arrows, carries over untouched. The extra dimensions are just room to file more nuance: more drawers, same cabinet.

And it doesn’t stop at words. A sentence, a document, an image, a sound, even a strand of DNA: anything you can compare can become a point. Better still, modern models file a photo and its caption in the same place (that is multimodal: one map for image and text). This is what makes search “by meaning” work: you type an idea, the machine encodes your query as a point and hands back its nearest neighbours. No keywords: geography. It is worth pausing on a quiet convergence here: neuroscience suggests the brain files concepts on the same kind of spatial map it uses to navigate rooms. Two systems that could not be more different (meat and silicon) landed on the same idea: meaning is a position in a space. That may not be a coincidence.

Where this stands in 2025–26

Train two AI models separately, on different data, with different architectures, and their spaces look different: axes rotated, scales changed. But look at the internal shape (which point is near which, the constellation of distances) and it is startlingly the same. This is the Platonic Representation Hypothesis (MIT, 2024): as models grow, they converge toward a shared “shape of reality,” as if there were one ideal map of meaning that each rediscovers in its own way.

In 2025 a Cornell team pushed the logic all the way (“Harnessing the Universal Geometry of Embeddings”, revised January 2026): they translated one model’s vectors into another’s with no paired examples at all, purely by making the two geometries coincide. Two consequences follow, one beautiful and one uncomfortable. The beautiful one: representations become portable: what one system learned can, in principle, be carried to another without deforming. The uncomfortable one: vectors stored “anonymously” are not anonymous. Whoever holds only the geometry can re-infer the content, enough to classify or extract attributes from documents they never saw. If you keep a database of embeddings, treat it like the sensitive data it encodes, not like meaningless noise.

Two more shifts worth knowing. First, the map now fits on a phone: Google’s EmbeddingGemma (September 2025) is an open embedding model built to run on-device, no cloud required, meaning you can host an entire “space of meaning” locally. Second, the Matryoshka trick (now standard in models from OpenAI, Gemini and EmbeddingGemma): a single vector whose tail you can truncate (keep the first 256 numbers instead of 768) and it still works, because the gist is filed at the front. That is a direct dial between precision and speed: the same map, at several resolutions, like a set of nested dolls.

Meaning, it turns out, was never in the words. It was in the arrangement.

Further reading

• The Illustrated Word2vec · Jay Alammar, the definitive visual introduction: the personality vector, cosine similarity, the king–queen analogy in pictures.
• Introducing EmbeddingGemma · Google DeepMind, the open, on-device embedding model; practical and hands-on.
• The Platonic Representation Hypothesis · arXiv, the argument that models converge to a shared “shape of reality.”
• Harnessing the Universal Geometry of Embeddings · arXiv, translation without a dictionary, and its security consequences.
• Matryoshka Representation Learning · arXiv, one embedding, many resolutions.