The Knowledge Spiral
Infinite universes and the optimism of creativity
John Gillespie Magee, Jr.’s poem High Flight captures the awe of aviation. He wrote it at age nineteen during his military service as a pilot in WWII. A month after its release, he died in combat. A first reading inspires awe at the miracle of human flight. A closer look reveals something deeper—optimism.
There is an inherently pessimistic worldview peddled across ideological spectrums. It is a worldview that is elegantly captured by a quote from the late Stephen Hawking, who said that humanity was nothing more than “chemical scum on a moderate-sized planet, orbiting around a very average star in the outer suburb of one among a hundred billion galaxies.” This reductionist, materialist, and hard deterministic view of humanity leads to all sorts of incorrect and incoherent thinking. Its biggest casualty is our misunderstanding of creativity.
Recently, I’ve focused on the increasingly serious idea that consciousness is fundamental to our universe. I believe that the rise of seriousness in consciousness research is one of the most exciting areas for humanity at the moment. But recently, I found it hopeful to discover an optimistic, albeit materialist, view of humanity that sees no issue with recognizing that we do not know where creativity comes from, but it does seem to make us unique. And that ability for humans to create new knowledge may be a sign for why free will exists.
Innovation
During the pandemic, I joined the founding team of aerospace startup. Our product was deceptively simple. It had been attempted before (unsuccessfully), and consisted of no new scientific breakthroughs. Instead, it was a unique combination of existing and proven technologies. Critics correctly noted the Soviets previously failed with a similar concept. The Soviet Caspian Sea Monster, which used ground-effect to fly close to the water, was noisy, unstable, hard to pilot, and flew too fast to be used as a passenger craft in crowded harbors. In contrast, our product used batteries (quiet), hydrofoils (agile), and flight control software (stable). The result was a completely new form of transportation that never existed before. A flying-boat-plane that could serve military and commercial customers and unlock hard-to-reach coastal destinations.
That unique combination, at that specific time in history, made it not only novel, but useful. New knowledge was created. When you’re in the middle of creating something new, it is easy to lose sight of the forest for the trees. At the time, I never took a step back to ask the bigger question—why did no one think of this before?
Once you understand how often this combination of existing materials turns into knowledge, you start to see it everywhere. Some innovations might have been accomplished much earlier in our species’ timeline had someone just thought to combine existing materials. As astrophysicist Peter Hague notes—
“Silver chloride and ammonia have been produced since antiquity, and the camera obscura is similarly ancient. The only real tech barrier to photography was a lens to allow enough light in to form an image - and Europe has been able to make them for about 800 years. Medieval science just overlooked the idea. If the right knowledge had been found out, we could now have photographs of Tudor soldiers displaying the body of Richard III, Columbus and his crew ready to depart, and London before the great fire. Mechanically powered moving pictures may well have followed, and you might be able to see silent film of the US founding fathers. That the available technologies were not combined for centuries is to me a catastrophic loss of information. [italics mine]”
Like adding hydrofoils to ground-effect planes, why didn’t someone think to put silver and ammonia together with a lens to create photography centuries before it eventually came to fruition? We don’t know. All we know is that if we look back on new knowledge creation, there seems to be a non-linear intuitive leap, or hunch, that someone experienced when faced with a problem, which ultimately led them down a rabbit hole of exploration, experimentation, and testing.
In Stalking The Wild Pendulum, Bentov wrote about how knowledge moves on an upward spiral that allows us to see from a broader perspective.
“Knowledge moves in an ever-expanding upward spiral, which allows us to see from the higher turns of the spiral our previous knowledge in a broader perspective. Thus, Newton’s mechanics have become a "special case” within Einstein’s theory of relativity. So, eventually, will Einstein’s theory of relativity become a "special case” in a science that will account for both physical and mental phenomena. “
— Itzhak Bentov
Once a new piece of knowledge is created it becomes automatic. It causes itself to remain so. None of us will ever need to re-invent the wheel. We will always know how to do so.
But where does knowledge originate?
David Deutsch’s Beginning of Infinity argues knowledge creation comes from either biological evolution or human thought—and uniquely, human creativity is unbounded. A visiting professor of physics at Oxford University, Deutsch is also the father of quantum computing and philosopher of science in the Karl Popper tradition.
In the biosphere, variation comes from mutations in genes and natural selection favors variants that most improve the ability of their organisms to reproduce and spread. In the case of human knowledge, the variation comes through conjecture1 and the selection is via criticism. Where evolution is bounded and local, people’s creativity is unbounded and has reach—they can extend to solving problems beyond those they were created to solve.
The uniquely significant difference between humans and other animals, Deutsch argues, is our ability to create new explanations that have reach. We are universal explainers. And that difference is the source of Deutsch’s optimism. The optimism that there is no problem in the universe that cannot be explained and solved by humans with sufficient knowledge.
Many Worlds
Science continues exploring challenging frontiers, such as the elusive Theory of Everything, which seeks to unify quantum mechanics and general relativity.2 Identifying the source of consciousness is another problem that seems to be getting more attention. There is this notion that if we understood what consciousness is and where it derives from, we might begin to understand the source of human creativity and its extension to creating true Artificial General Intelligence.
In my previous essay I mentioned that materialists tend to recoil at the word ‘consciousness’ because a) there isn’t an agreed upon definition, and b) to them it implies something supernatural. Even the most robust consciousness theories that are measurable get labeled pseudoscience. There is so much institutional resistance to anything that might smell of woo. By far, one of the biggest hurdles the field of consciousness research has is how it relates to quantum theory.
Quantum Theory is the branch of physics theory that seeks to explain phenomena occurring at an atomic, or even subatomic, scale. Quantum objects simultaneously behave as waves and particles, their observed form dependent on measurement. Instead of an atom being a definite particle with a definite position and momentum, it is more like a wave function of probability. And the standard view is that of all the probabilities, it collapses into one single position only once it is measured. The problem is no one ever seems to agree on what measurement means. This is often called the measurement problem. This two minute explainer from Sean Carroll is worth the watch to get up to speed:
David Deutsch sees no point in dancing around what it might mean for a wave function to collapse because of the action of an observer. He subscribes to the Everett, or Many-Worlds, interpretation of quantum theory. In his view, calling it an "interpretation" is already a misnomer. As he explains, Everett’s formulation is simply quantum mechanics taken literally, at face value, without adding unnecessary metaphysics. Its just quantum mechanics without the extra assumption of wave-function collapse. The universal wave function evolves into multiple distinct universes, each corresponding to one of the possible outcomes. The state you and I observe is just one of these branching universes.
Instead of Schrödinger's Cat being alive and dead until you open the box, the poor cat is both dead and alive in different universes. Follow the logic and you now have infinite universes branching at each quantum event. If you want to read a wonderful short story (fiction) that follows the Many-Worlds Interpretation all the way through, give this one a read.
The idea that at different decision points, the universe branches off such that there are all these different versions of you in many worlds should either fill you with wonder or terror. For Deutsch, it's unequivocally the former.
Deutsch's principle of optimism states that every problem not forbidden by the laws of physics is solvable, provided we create the explanatory knowledge needed to solve it. Progress hinges on our unique human ability as universal explainers to create explanatory knowledge. This means the agency and responsibility to solve problems lie with us. From his multiverse perspective, the resources and possibilities for solving problems are vast, but progress depends on our ability to create the necessary knowledge. And since humans are universal explainers with the unique capacity to create knowledge, the agency and choice to create new knowledge belong to us. Deutsch’s optimism is not because another universe solves our problems, but because we ourselves can solve them by creating explanatory knowledge in this universe.
Contrary to what reductionists say, humans are not mere machines or chemical scum. Deutsch and Carroll—though materialist physicists—see no contradiction between determinism and meaningful free will. Our agency and choice arise directly from our ability to create explanatory knowledge about possible outcomes.
It follows logically: the optimal path is to prioritize good choices and our extraordinary capacity for knowledge creation. As the MWI tells us, we will always subjectively experience only one of the possibilities of our choices, so we might as well work to ensure it's the best one.
Whether it is in choosing a career, taking your passion seriously, or improving ourselves, our communities, and our world, we can do all of these things because we are humans—universal knowledge creators navigating through the infinite branches of the multiverse with the power of explanation and problem-solving that Deutsch argues makes us cosmically significant.
As I made final edits, a note on Substack popped up from my friend Dylan—a reminder of why optimism matters:
The growth of Knowledge consists of correcting misconceptions in our theories. Ever since the Enlightenment this had happened through a tradition of criticism. New explanations are proposed, criticized, and then refined.
Since the tradition of Conjecture and criticism has created all Knowledge and Progress it's absolutely imperative that we keep it. Free speech is critical. No idea should be beyond criticism.