Why can we add words together? Anyone who has done enough reading can take two words, “add” their semantic meanings together in their heads, and find a new word that is representative of their combination, or else invent one to designate the synthesis. The words themselves might have some more global structure within them such as logic, but in this way they locally behave like a vector space. This certainly fits the definition of a “manifold”: a topological space that is locally homeomorphic to a Euclidean space. However, by itself this does not truly answer the question of “Why?”
On the one hand, it would be naive to think that this is an intrinsic structure of the universe; there are many phenomena that are better modelled with mathematical components that do not lend themselves to a manifold structure, such as singularities, discontinuities or discrete structures. On the other hand, it is difficult to understate the unreasonable effectiveness of manifold learning algorithms.
What is Manifold Learning?
Allegory transforms the phenomenon into a concept, the concept into an image, but in such a way that the concept remains nevertheless still contained in the image so that it can be entirely held and possessed and expressed in it. The use of symbols transforms the phenomenon into an idea, the idea into an image, and in such a way that the idea still remains infinitely active and accessible in the image so that, even expressed in all languages, it remains inexpressible.
Goethe, Nachlass
Suppose you have some extremely high-dimensional data from the real world, such as the spatial coordinates of a large set of sensors measuring a mechanical system, or a dataset of images of animals. The Manifold Hypothesis states that since it was taken from the real world, it will not actually fully utilise all of those dimensions. Instead, there will exist some latent manifold embedded within that space that the data really lies on.
Manifold learning is primarily concerned with finding this latent manifold; find the manifold and you have described the system, potentially in a much more intuitive way than if you had used some brute-force method. Directions could start to correspond to meaningful semantics within the system: in a mechanical problem this is frequently cited as some degree of freedom, and in a natural language context it could be a high-level concept such as the language used, for example English, or even particular sentence structures.
Modern neural networks can be interpreted as dealing with manifold structures, but using multiple layers. A set of layers with transformations between them was the basis of the first successful neural network: the multilayer perceptron. This was a big step up from a single layer model, which famously cannot simulate a XOR gate. Each layer can be interpreted as a coordinate system for a manifold, and weights between each coordinate represent a local transformation from one set of coordinates to another. In the structure of trained neural networks a pattern emerges: ID, or intrinsic dimensionality, starts off low, spikes midway through the model, and then decreases again. This local maximum in ID represents the lowest dimensional manifold representation of the data and can be shown to contain the most semantic meaning.
Consider the sequence of weighted transformations that take place between the manifolds embedded in each layer of the network. The rapid expansion of dimensionality is the model amplifying those features that it deems important. It then rearranges these features into a more tractable format in subsequent layers before shrinking them down to maximise computational efficiency. Finally, the dimensionality has to either expand or shrink to fit the output format, such as with a high dimensional token space or a one-dimensional classifier. Each transformation from layer to layer is a representation of universal computation, and in each layer the embedded manifold is precisely the envelope of the training data.
Conclusion: So Why Can We Add Words?
The solution is to consider cognition itself; if machine learning algorithms are slowly starting to mirror how we think, this makes sense. We think in manifolds, and our mathematics is based around continuous notions, so our vision of machine learning is greatly enabled by new geometric insights. Any set of coordinates we impose on data in measurement is likely going to be inefficient compared to our internal representation, and so we can justify the manifold hypothesis.
Sources
- Transformers on Manifolds
- The geometry of hidden representations of large transformer models
- Attention is All You Need
- Natural Gradient Methods: Perspectives, Efficient-Scalable Approximations, and Analysis
Appendix: A Comparison of Dimensionality Measuring Algorithms
T-SNE
Purpose, strengths and weaknesses: documented in scikit as primarily a tool for visualising data, so may not be applicable for an interpretability algorithm.
Mechanism: points are interpreted as measurements of a probability distribution, and Kullback-Leibler cross entropy is minimised.
Results: most trustworthy, or best output; third best time to run, half as fast as the next.
Metric MDS
Purpose, strengths and weaknesses: aims to reproduce pairwise distance.
Mechanism: metric MDS preserves distance magnitude, non-metric MDS preserves distance ordering.
Results: second best trustworthiness, but with significantly longer running time.
Spectral Embedding
Purpose, strengths and weaknesses: fastest algorithm by far, good for curved and disconnected structures. It cannot embed new points after, so may end up being less useful for some interpretability workflows.
Mechanism: constructs a neighbourhood graph and uses eigenvectors of the Laplacian.
Results: created a notably distorted graph, but maintained a surprisingly high trustworthiness rating similar to Metric MDS. Vastly outperformed others in speed.
Modified LLE
Purpose, strengths and weaknesses: preserves local neighbourhood geometry.
Mechanism: LLE creates a vector for each neighbourhood, and optimises to find the best set to project onto a lower dimension. Modified LLE uses multiple linearly independent weighted vectors at each neighbourhood instead of one.
Results: worst performance, decent runtime. It gets beaten by the spectral algorithm in both cases, so the tradeoff is not optimal.