And the mechanics of Information Theory.
Research suggests that the first single-celled organisms formed around 3.8 billion years ago but it was not until about 600 million years ago that, quite suddenly in evolutionary time, the Cambrian explosion occurred, giving rise to multicellular life, diversification and biological complexity1. And you and me. So what is a complex system? A simple definition is that it is a system with agent-like objectives and action properties that cannot be predicted from its parts alone. This complexity has puzzled humanity since at least the time of the Greeks and more recently entered the domain of scientific enquiry in the second half of the 20th century. Indeed, the famous Santa Fe Institute was established in 1984 by leading luminaries from science specifically as a research body to study the science of complex adaptive systems (and to weaponize them of course, apart from other objectives). How do simple elements converge in seemingly objective and agent-driven ways to form complex and dynamic biological systems such as cells, tissues, organs, brains and minds? There are many competing theories and many appear to be making the point that whilst classical Neo-Darwinist evolution and random selection is necessary, it is not sufficient. Why? Because the gene-to-RNA-to-protein one-way search-space paths are probably too computationally large, and the ratio of adaptive-to-non-adaptive mutations is too high to allow for living systems to have evolved quickly enough in the geological time available, given out current understanding of the biological systems processes involved. Some are proposing that we need a better understanding of existing biological signalling and production systems such as epigenetics, and yet new systems. And some are outlandishly even calling for a reassessment of our understandings of the fundamental physics of reality and spacetime in order to account for the dichotomy; a new model that would transcend the spacetime model of general relativity and the standard model of quantum mechanics.
Biological neural networks can be thought of as a collection of weak learners chatting with each other. Individual neurons have limited predictive power or control over the organisms’ environment yet agent-like planning and control is manifest and multilevel from cell upwards, whether the organism be a sea-slug, a professional footballer or a professor. Applying reductionism to the system does not yield the output. The whole is more – a whole lot more – than the sum of the individual parts, even accounting for interconnectivity. This is sometimes described as the observability problem: If it were possible to observe the full system, then the behaviour would, by Laplacian logic, be understandable and predictable2. But we cannot, and the problem appears intractable. Thus scientists revert to machine-learning models and watch and tinker with their parameters until the model and the reality iteratively converge in output, based on input.
It is necessary to understand a key point regarding information content and patterning: Information in the brain for example does not reside simply at the level of inter-neuronal connections and weights, it appears to exist, to my imperfect mind, multidimensionally, hierarchically and in superposition through the myriad and conceptually infinite dynamic network of engram patterning that is produced. As far as I can tell, and this may appear to be a somewhat large speculative leap, it exists in dynamic multidimensional (Hilbert) information space. This space may be multifariously instantiated in physical matter and spacetime but is exists before, above, below and after material spacetime. And interleaves with it; an eternal golden braid, as Hofstadter noted, twelve years after the summer of love.
The concept of information patterning – of information theory – is tricky to understand and to visualise but is worth it. Classical machine learning uses the concept of multiple interconnected sparse matrices that interact with each other. A better way to think about it is as dynamic informational networks interacting in multidimensional information space that sometimes spontaneously fire together according to (non-causative) correlations in this information space. Hence the suddenness of our ‘hah’ and our ‘eureka’ moments. Personally, I find that these often happen in a wavelike succession of developing and self-tuning intuition, presumably because the neuronal mathematics is operating in iterative forward-backward recursive Bayesian-like passess over the internal model data and as new inputs are received. Maybe this is why, despite the brain processing inputs from distinct sensory modalities we perceive entities as a whole, because the data is combined in information space not squishy clumps of neuronal tissue. And as a corollary, despite this informational connectedness, there will be no obvious neuronal connectivity except possibly within the EM and other information field spectra (yes, this does seem to hark back to 19th century vitalism, but it is more than that). The way that the brain connects this integration of distributed information streams is known as the binding problem. We also have a measurement challenge here in that is appears useful to measure the informational size of causal emergence and the ratio of this macro-level phenomenon to the constituent parts. One might call this the Complexity Ratio of the organism. Or the negentropy ratio.
Scientists are attempting to develop mathematical models for this emergent adaptive complexity. This requires the development of a functional architecture. And one might attempt to develop a classical logical decision structure based on the von Neumann model of computation. However this is patently not possible with a hierarchical dynamic neural network such as the human brain or an artificial neural network (ANN). This point was vividly demonstrated for example when Deep Mind’s Alpha Go beat the human World Grand Master, Lee Sedol: Mid-game the computer started making unexpected moves that no human could understand – and which the machine obviously could not explain itself – but which led to winning outcomes. It took humans some time to decode and understand the strategic and tactical nature of these moves. This is the nature of a deep neural network. The mathematics comes first, the logic of Aristotle and von Neumann follows.
The problem though is that mathematicians can construct any model they like to mimic a physical system and they can iterate until the behavioural and functional deltas between model and reality is minimised. But the model is not the organism. The map is not the territory. The mathematics is not the physics. Einstein himself famously said once in this context that “Since the mathematicians have invaded the theory of relativity, I do not understand it myself anymore”. It is arguably a barren pursuit to apply what we used to call numerical methods and Operational Research techniques to a problem without first having a visceral and ideally a visual understanding of the nature and physics of the problem. Intuitive guesses are good though.
Another speculative corollary of the information theory I have outlined above is what I call Shannon Now, as I have touched on before. Our consciousness, again as far as I can tell after thousands of miles of dog-walking, exists in Shannon Now. As such it is at least partially independent of matter and spacetime. A kind of physics-based neo-Vitalism. For example early this morning I heard the wind-gust clatter of a window and it immediately brought into my conscious mind the sights and sounds and smells and colours of a childhood holiday. It is as if the now of my conscious awareness and my subconscious memories sparked together in a quick dance. Converting this rhetoric into more scientific prose it is as if the engram of my conscious ‘now’ spontaneously connected with iterative waves of conformationally-changing subconscious memories, each wave yielding a subtly different snapshot of multi-modal sensory perception, together also with all sorts of emotions across different axes. Like a stop-motion animation movie. Mathematically I picture this (sic) as pure information existing as a fractal and foldable network of nodes and links in dynamically dimensional Hilbert space. Each node opens, expands and disappears in a Markovian froth, under the law of what I call Information Gravity. But this is not just one gravity, it is gravities of mathematical correlations across information factors. It is axiomatic that these correlations must change, otherwise we would exist in a block (information) universe. So what causes these changes? These changes are caused, just like in a black or grey box box processor, by external signal updates that in turn generate internal micro-network or engram updates3 . Sometimes these changes are small and perturbational, simple mean-reverting changes maybe modulating on an underlying signal. Sometimes the conformational change, in my information space, is revolutionary and spontaneous and extensive – the aha moment. Or sometimes it is just a nice memory of a long-past joy.
This all has interesting implications, for my own philosophy of life and meaning if nothing else. More ambitiously and speculatively it suggests to me, and I think many others, that it is time to complement, reframe and on-develop our understanding of life and universe as explained by classical mechanics, general relativity, the standard model and (forever incomplete) quantum mechanics with a new model; a model of information mechanics. I call this Quantum Information Reality (QIR), others will have other and probably better names for it.
More on this to come, dog-walking and poetry and football allowing.
1 It could though be argued that this biological (and mathematical) complexity was already there with the first prokaryotic cells that, as far as scientific investigation can tell so far, seemed to emerge ‘fully formed’ soon after the earth was cool enough to carry such life. Some call this the black hole of biology.
2 This is a good thing btw as 100% predictability would remove Free Will, a concept I am quite keen on.
3 One albeit simplistic way to visualise this is as a kind or informational origami in which one concept folds into another concept, or subtly updates into a richer meaning.
