Coherence Machines
Intelligence as Geometry, Constraint, and Coordination
Executive Summary
Every major AI architecture, from Hopfield networks to transformers, relies on a foundational assumption: the geometry of the system remains largely fixed while computation moves through it.
Weights are trained. The landscape stabilizes. Inference begins. The structure holds still.
This assumption enabled extraordinary progress. Fixed geometry made optimization tractable, scaling possible, and representation stable. But modern systems are beginning to strain against its limits.
To a certain extent, retrieval-augmented generation, tool use, agent orchestration, dynamic routing, and external memory systems are already expanding cognition beyond a fixed set of parameters. In practice, the actual geometry of these systems is no longer strictly static. It is distributed, dependent on context, and reconfigured to some degree during operation.
Biology indicates that this is not an isolated phenomenon, but rather a fundamental pattern. DNA does not function as a rigid, unchanging program. Instead, it serves as a stable foundation embedded within multiple layers of regulation, including epigenetic, metabolic, neural, and environmental factors. Furthermore, this program operates simultaneously across multiple timescales.
Intelligence, from this perspective, doesn’t come from a single system. Instead, it arises from the interaction of multiple systems. These systems operate across different geometries, timescales, and constraints.
A similar trend is showing up across AI. Approaches like Liquid Neural Networks, Neural ODEs, Sparse Mixture-of-Experts, Active Inference, world models, neuromorphic hardware, and external memory systems to name a few. Each of these approaches ease parts of the fixed-geometry assumption in some way.
The key question now is no longer just how to scale up a single architecture. Instead, the focus is shifting to coordinating and integrating different types of computational systems. The main challenge is figuring out how these systems adapt and evolve under different constraints.
The biggest change happening now isn’t just switching from one way of thinking to another. It’s about creating systems that combine different computational regimes into smooth, multi-scale architectures. A Langlands style approach to intelligence so to speak.
This essay was partially catalyzed by recent discussions surrounding Daniel Rodriguez’s reflections on evolving geometries in AI and thoughtful critiques raised by Denis O. regarding the distinction between frozen parameter manifolds and runtime dynamics. What began as a technical disagreement increasingly appears to point toward a much deeper architectural question.
For decades, artificial intelligence has been built upon a deceptively simple assumption:
The structure of the system remains fundamentally fixed while computation moves through it.
The assumption is so deeply ingrained in modern machine learning that it’s rarely stated outright by researchers. However, it underlies nearly every major architecture in AI, ranging from traditional neural networks to modern transformers.
Weights are trained. Geometry stabilizes. Inference begins.
The landscape holds still while the process moves.
This assumption was not irrational. In many ways, it was necessary. Without stable geometry, modern optimization would likely become computationally intractable. Fixed representational spaces helped researchers in several ways. They defined gradients, stabilized training, compressed memory, and scaled systems to new heights.
The result has been extraordinary progress.
But there is a growing sense that something important is beginning to strain against this paradigm.
The issue goes beyond just scale. It’s not just a matter of bigger models, larger context windows, or better hardware. The problem runs deeper, involving fundamental architectural and philosophical concerns. It begs deeper questions:
What if fixed-geometry computation is only one component of intelligence and not sufficient on its own?
What if intelligence comes from how stable structures interact with the processes that control, shape, and coordinate them over time?
These questions may ultimately matter more than parameter count or scaling laws.
What Do We Mean by “Geometry”?
Before going further, it is important to define the term carefully.
In this essay, “geometry” does not merely refer to physical space. It describes the “structure” governing how information is accessed and shared in a cognitive system. This includes interaction costs, memory locations, routing paths, activation patterns, and overall connectivity.
Modern AI systems already operate across several overlapping forms of geometry:
This distinction matters because the debate emerging inside AI is no longer simply about models. It is increasingly about which geometries remain fixed, which become adaptive, and how constraints regulate their evolution across time.
The Legacy of Hopfield
To understand why this matters, it helps to revisit one of the foundational ideas in neural computation: the Hopfield network.
John Hopfield pioneered a compelling framework for understanding memory and cognition by applying physics principles. By conceptualizing memories as attractors in an energy landscape, he revealed how the system progresses toward stable states through a process of energy minimization. This innovative perspective transformed cognition into a navigational process, where the mind traverses a predetermined landscape to reach optimal outcomes.
The weights encoded the topology of the landscape itself. Once trained, the geometry remained largely fixed while states evolved dynamically within it.
This distinction became foundational: geometry stable, activation dynamic.
Modern deep learning inherited this assumption almost completely. Even today’s transformers largely operate this way.² During inference, weights remain fixed:
Attention mechanisms actively control the flow of information, but they do not alter the underlying structure of the learned parameters during operation. This key difference was highlighted in a recent discussion sparked by Daniel Rodriguez’s article on evolving geometries and intelligence, where commenter Denis O. pointed out a crucial technical flaw: contemporary deep learning systems do not modify their fundamental manifolds during the inference process.
Attention traverses the space; it does not reconstruct the space itself.
Technically, he is correct.
But Rodriguez may also be pointing toward something real.
And that tension reveals a deeper fault line emerging inside AI.
Runtime Geometry
The real issue may not be whether weights change during inference. The real issue is whether the effective cognitive geometry of the system remains static.
Modern AI systems increasingly derive capability not merely from stored parameters, but from dynamic routing, retrieval, memory orchestration, tool interaction, context construction, recursive prompting, external coupling, and runtime activation topology.
RAG, tool-use systems, agent orchestration layers, and external memory architectures push parts of cognition outside the pretrained model. The “mind” of the system increasingly extends across model, memory, routing, tools, environment, and runtime coordination.
In other words: the operational geometry of the system is already becoming more dynamic than the frozen weights beneath it.
Attention mechanisms hint at this shift. A transformer does not process every token equally. Attention dynamically alters relational adjacency, salience, and informational accessibility. Different prompts instantiate different temporary geometries over the same underlying substrate.
The parameter space may remain frozen. But the runtime topology does not.
This suggests an expansion instead of a clean switch from one paradigm to another. We are moving from systems that use fixed parameter geometry to those that mix stable representations with dynamic, context-sensitive geometries created at runtime.
In this view, runtime geometry does not replace learned structure. It augments and organizes it. Selectively activating, routing, and composing capabilities in ways that static representations alone cannot support.
Biology Never Operated on Frozen Geometry
Biology provides an important counterpoint.
DNA is often described as a blueprint or static program, but this framing is deeply incomplete. Biological systems do not merely execute fixed instructions. They continuously regulate accessibility, expression, timing, and coupling across multiple interacting layers.
The genome provides a deeply stabilized substrate shaped through evolutionary hysteresis.
Intelligence and adaptation come from dynamic regulation on top of it. This includes:
Epigenetics
Cellular signaling
Hormonal modulation
Neural activation
Synaptic plasticity
Environmental feedback
Developmental timing
The organism is not simply “running code.” It is continuously reshaping which pathways become accessible, salient, or suppressed under changing environmental constraints.
Importantly, these processes operate across radically different timescales simultaneously. A neuron fires in milliseconds. Synapses adapt over minutes or hours. Epigenetic states may persist for years. Evolutionary adaptation unfolds across generations.
Intelligence in biology is not merely computation. It is multi-temporal regulation across nested geometries under energetic and environmental constraints.
The parallels to emerging AI systems are becoming increasingly difficult to ignore:
This does not mean AI systems are “becoming biological.” It means biology may reveal architectural principles that static computation alone does not capture well.
Researchers such as Michael Levin have explored how cognition, memory, and goal-directedness emerge across multiple biological scales.⁴ Similarly, Karl Friston’s active inference framework emphasizes adaptive regulation under energetic and informational constraints.⁵ These ideas are no longer isolated philosophical curiosities. They are beginning to converge with emerging AI architecture questions.
Emerging Architectural Signals
These ideas are not emerging from one isolated research direction. Several independent lines of work are easing the frozen-geometry assumption. These research efforts aim for more dynamic, constraint-aware cognition that is regulated by geometric principles.
Liquid Neural Networks introduce continuously adaptive internal dynamics that allow temporal structure to reshape processing in real time.⁶ Neural ODEs replace discrete layers with continuous-depth evolving trajectories, treating computation itself as a dynamical system.⁷ Sparse Mixture-of-Experts architectures demonstrate that dynamic routing under resource constraints can outperform uniformly dense computation.⁸ Active Inference systems regulate uncertainty and action selection under bounded resources through free-energy minimization.⁵ World models and Joint-Embedding Predictive Architectures (JEPA) build latent predictive environmental structure that enables anticipation rather than mere reaction.⁹ Neuromorphic systems pursue event-driven, energy-aware computation that reintroduces physical constraint as architectural principle.¹⁰ RAG and external memory architectures distribute cognition across memory environments beyond the model boundary.³ SeedIQ, Denis O.’s own active-inference framework, uses geometric metacognition and distributed constraint satisfaction to achieve ARC-AGI-style reasoning without traditional scaling or large language models.¹¹ Emerging holonomic architectures (UREIL 3B, J. Harlow) emphasize semantic geometry, manifold curvature, and extreme parameter efficiency through geometric continuity rather than brute-force parameterization.¹²
These systems collectively reveal a crucial insight: researchers focusing on dynamic routing, embodiment, active inference, environmental coupling, world modeling, energy efficiency, and adaptive orchestration are converging on a similar idea.
Intelligence likely arises more from adapting to changing relationships within specific boundaries than from relying on fixed representations.
The Limits of Pure Scaling and the Return of Constraints
This convergence may help explain why current AI systems increasingly exhibit a peculiar asymmetry: extraordinary local capability paired with limited long-horizon coherence.
Modern models can solve highly specialized problems with remarkable fluency. Yet they often struggle with persistent identity, stable long-term reasoning, adaptive contextual continuity, multi-scale planning, and coherent developmental learning. The issue may not simply be insufficient scale. It may be that static architecture is approaching the limits of what frozen geometry can support.
One of the strangest features of modern computing is how effectively it hides physical reality. Software behaves as if movement were free, memory appears globally accessible, latency feels abstracted away, and energy costs disappear behind the interface. But physical systems do not work this way.
Every biological intelligence is shaped by metabolic limits, signal propagation delays, communication bottlenecks, thermal constraints, and environmental pressures. Constraint is not an obstacle to intelligence. Constraint is what gives intelligence shape.
This may become one of the defining shifts in next-generation AI architecture. Rather than treating constraints as engineering nuisances to eliminate, future systems may increasingly treat them as active geometric regulators. Instead of asking “How do we maximize computation?” we may begin asking “How do we regulate adaptive geometry under bounded resources?”
That is a profoundly different design philosophy. Energy-aware training, sparse routing, latency-aware orchestration, neuromorphic hardware, and software-defined reconfigurable systems may all represent early manifestations of this transition.
The key thing to remember: no single computational paradigm currently explored in AI—statistical, symbolic, geometric, or dynamical appears sufficient on its own to support general, adaptive intelligence across contexts and timescales.
From Static Computation to Coherence
This shift has implications far beyond neural networks.
It suggests that intelligence may not be based on symbolic manipulation, parameter retrieval, or static optimization. Intelligence may arise from regulating coherence in dynamically evolving geometries. These changes happen over different timescales that interact, all under energy and environmental limits.
In this framing, memory changes, learning changes, hardware changes, identity changes, cognition changes. The boundary between software, environment, memory, and inference begins to blur. The system no longer merely processes information. It navigates and reshapes its own constraint topology.
Beyond Architectural Monocultures
A central risk in today’s AI discourse is replacing one monoculture with another. As the limits of transformer-based systems become clearer, it is tempting to search for a single successor paradigm—world models, active inference, symbolic reasoning, neuromorphic systems, or geometric cognition.
Biology points in a different direction.
Higher cognition does not appear to arise from a single computational regime repeated at scale, but from the coordination of multiple specialized systems operating across different timescales and constraints.
The brain can be understood as a layered ecology of partially specialized subsystems:
sensory cortex: probabilistic and predictive processing
motor systems: dynamical, control-oriented computation
hippocampus: relational and spatial memory
cerebellum: timing and error correction
language systems: hybrid symbolic–statistical processing
basal ganglia: action selection and reward gating
default mode network: simulation and world modeling
glial and metabolic systems: energetic regulation
These systems are not interchangeable, and they do not operate at the same temporal or energetic scales. Higher cognition emerges from how they coordinate.
This perspective helps explain why many emerging AI approaches appear simultaneously promising:
Transformers excel at large-scale statistical abstraction. World models capture latent predictive structure. Diffusion models demonstrate iterative generative refinement. Neural ODEs and Liquid Networks explore continuous dynamics. Geometric deep learning emphasizes relational structure. Active inference focuses on regulation under uncertainty and constraint. Neuromorphic systems foreground energy and event-driven computation.
Seen this way, these approaches are less like competing answers and more like distinct computational dimensions.
The question is no longer: “What replaces transformers?”
But: “How do heterogeneous computational regimes coordinate coherently across multiple timescales, geometries, and constraints?”
The Next Architectural Question
If this trend continues, the future of AI won’t just be about bigger models. Instead, it will focus on creating complex systems that coordinate and connect different types of computation. These systems will need to manage interactions between different computational regimes, taking into account factors such as time, memory, energy, and environment, and leveraging technologies like runtime geometric regulation and adaptive routing. Rather than building fully self-modifying systems, the field is moving toward architectures that seamlessly integrate probabilistic, symbolic, and geometric computation.
The brain serves as a powerful model for this approach. It combines various computational methods and systems to create complex behaviors. This happens through a system where processes interact and coordinate with each other. This approach allows the brain to operate effectively across different scales and timescales, and to adapt to changing conditions and constraints. Future AI systems must similarly integrate multiple forms of computation while balancing competing constraints to enable such complex behaviors.
The key to developing these systems will be to understand how to create architectures that can support the integration of diverse computational approaches, and that can adapt and evolve over time in response to changing conditions and requirements. By pursuing this goal, researchers can create AI systems that are more sophisticated, flexible, and effective, and that are better able to tackle complex real-world problems.
My own early take on what shape this may take:
The brain is not one algorithm repeated at scale, but a coordinated system of interacting processes distributed across multiple geometries and timescales.
Artificial systems may evolve in a similar direction—not toward a single dominant paradigm, but toward coherent orchestration across heterogeneous forms of computation.
The central challenge, then, is not simply building more powerful models.
It is learning how to build systems in which many kinds of intelligence can coexist, interact, and remain coherent under constraint.
My own early take on this:
Notes & References
Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79(8), 2554–2558. DOI: 10.1073/pnas.79.8.2554
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems, 30. arXiv: 1706.03762
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems, 33. arXiv: 2005.11401
Levin, M. (2019). The Computational Boundary of a “Self”: Developmental Bioelectricity Drives Multicellularity and Scale-Free Cognition. Frontiers in Psychology, 10, 2688. DOI: 10.3389/fpsyg.2019.02688
Parr, T., Pezzulo, G., & Friston, K. J. (2022). Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press. DOI: 10.7551/mitpress/12441.001.0001
Hasani, R., Lechner, M., Amini, A., Liebenwein, L., Ray, A., Tschaikowski, M., Teschl, G., & Rus, D. (2021). Liquid Time-constant Networks. Proceedings of the AAAI Conference on Artificial Intelligence, 35(9), 7657–7666. arXiv: 2006.04439
Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018). Neural Ordinary Differential Equations. Advances in Neural Information Processing Systems, 31. arXiv: 1806.07366
Fedus, W., Zoph, B., & Shazeer, N. (2022). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. Journal of Machine Learning Research, 23(120), 1–39. arXiv: 2101.03961
LeCun, Y. (2022). A Path Towards Autonomous Machine Intelligence. Version 0.9.2. Meta AI Technical Report. Available at: openreview.net/pdf?id=BZ5a1r-kVsf
Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., Dimou, G., Joshi, P., Imam, N., Jain, S., Liao, Y., Lin, C.-K., Lines, A., Liu, R., Mathaikutty, D., McCoy, S., Paul, A., et al. (2018). Loihi: A Neuromorphic Manycore Processor with On-Chip Learning. IEEE Micro, 38(1), 82–99. DOI: 10.1109/MM.2018.112130359
Denis O. / SeedIQ. Distributed adaptive orchestration and bounded autonomy via active inference and geometric metacognition. LinkedIn activity reference. (Note: If a preprint or technical report exists, add DOI/arXiv link here.)
Harlow, J. / URIEL-3B. Holonomic architecture emphasizing semantic geometry, manifold curvature, and parameter efficiency. LinkedIn activity reference. (Note: If a preprint or technical report exists, add DOI/arXiv link here.)
Bronstein, M. M., Bruna, J., Cohen, T., & Veličković, P. (2021). Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges. arXiv: 2104.13478
Levin, M. (2022). Technological Approach to Mind Everywhere: An Experimentally-Grounded Framework for Understanding Diverse Bodies and Minds. Frontiers in Systems Neuroscience, 16, 768201. DOI: 10.3389/fnsys.2022.768201