A New Kind of Science

Categories: 2002 books | Science books | Controversial books

A New Kind of Science is a controversial book by Stephen Wolfram, published in 2002. It introduced and justified the empirical systematic study of simple programs, which are basic deterministic systems which iterate. Wolfram argues that the scientific philosophy and methodology appropriate for the study of simple programs is relevant to other fields of science. The book is available online (see links below).

Computation and its Implications

The thesis of A New Kind of Science is twofold: that the nature of computation must be explored experimentally, and that the results of these experiments have great relevance to understanding the natural world.

Since its crystallization in the 1930's, computation has been primarily approached from two traditions: engineering, which seeks to build practical systems using computation; and mathematics, which seeks to prove theorems about computation.

Wolfram describes himself as introducing a third major tradition, which is the systematic, empirical investigation of computational systems for their own sake. This is where the "New" and "Science" parts of the book's title originate.

However, in proceeding with a scientific investigation of computational systems, Wolfram eventually came to the conclusion that an entirely new methodology is needed. Traditional mathematics was failing to meaningfully describe the complexity seen in these systems. Through a combination of experiment and theoretical positioning, the book introduces a new methodology that Wolfram argues is the most realistic way to make scientific progress with computational systems.

This difference in methodology casts A New Kind of Science as a "kind" of science, and allows its principles to be potentially applicable in a wide range of fields.

The World of Simple Programs

The basic subject of Wolfram's "new kind of science" is the study of simple abstract rules — essentially, elementary computer programs. Traditionally, we think of computer programs as very complicated systems that are constructed for a purpose. But it is possible to look at the space of possible simple computer programs, and simply ask what they do.

Wolfram's fundamental experimental result is that given almost any class of computational system, one very quickly finds instances of great complexity among its simplest cases. This seems to be true regardless of the components of the system and the details of its setup. Systems explored in the book include cellular automata in 1, 2 and 3 dimensions, mobile automata, Turing machines in 1 and 2 dimensions, several varieties of substitution and network systems, primitive recursive functions, nested recursive functions, partial differential equations, combinators, tag systems, register machines, reversal-addition and a number of other systems.

A second and even more surprising experimental result is that even as the rules become more complicated in setup, their computational sophistication — and the variety of possible behaviors — does not seem to increase.

Mapping and Mining the Computational Universe

Given that very simple rules often do very complex things, how do we study them? Wolfram believes it is necessary to systematically explore all of these computational systems and document what they do. He believes this study should become a new branch of science, like a physics or a chemistry. The basic goal of this field is to understand and characterize the computational universe using experimental methods.

The proposed new branch of scientific exploration admits many different forms of scientific production. For instance, qualitative classifications like those found in biology are often the results of initial forays into the computational jungle. On the other hand, explicit proofs that certain systems compute this or that function are also admissible.

There are also some forms of production that are in some ways unique to this field of study. For instance, the discovery of computational mechanisms that emerge in different systems but in bizarrely different forms. Another kind of production involves the creation of programs for the analysis of computational systems — for in the NKS framework, these themselves should be simple programs, and subject to the same goals and methodology.

An extension of this idea is that the human mind is itself a computational system, and hence providing it with raw data in as effective way as possible is crucial to research. Wolfram believes that programs and their analysis should be visualized as directly as possible, and exhaustively examined by the thousands or more.

Since this new field concerns abstract rules, it can in principle address issues relevant to other fields of science. However, in general Wolfram's idea is that novel ideas and mechanisms can be discovered in the computational universe — where they can be witnessed in their clearest forms — and then other fields can pick and choose among these discoveries for those they find relevant.

Systematic Abstract Science

While Wolfram promotes simple programs as a scientific discipline, he also insists that its methodology will revolutionize essentially every field of science.

The basis for his claim is that the study of simple programs is the most minimal possible form of science, which is equally grounded in both abstraction and empirical experimentation. Every aspect of the methodology advocated in NKS is optimized to make experimentation as direct, easy, and meaningful as possible — while maximizing the chances that the experiment will do something unexpected.

Just as NKS allows computational mechanisms to be studied in their cleanest forms, Wolfram believes the process of doing NKS captures the essence of the process of doing science -- and allows that process's strengths and shortcomings to be directly revealed.

Wolfram believes that the computational realities of the universe make science hard for fundamental reasons. But he also argues that by understanding the importance of these realities, we can learn to leverage them in our favor. For instance, instead of reverse engineering our theories from observation, we can simply enumerate systems and then try to match them to the behaviors we observe.

A major theme of NKS style research is investigating the structure of the possibility space. Wolfram feels that science is far too ad hoc, in part because the models used are too complicated and/or unnecessarily organized around the limited primitives of traditional mathematics. Wolfram advocates using models whose variations are enumerable and whose consequences are straightforward to compute and analyze.

Philosophical Underpinnings

Wolfram believes that one of his achievements is not just exclaiming, "computation is important!", but in providing a coherent system of ideas that justifies computation as an organizing principle of science.

For instance, Wolfram's concept of computational irreducibility -- that complex computations cannot be short-cutted or "reduced", is ultimately the reason why computational models of nature must be considered, in addition to traditional mathematical models.

Likewise, his idea of intrinsic randomness generation -- that natural systems can generate their own randomness, rather than using chaos theory or stochastic perturbations -- implies that explicit computational models may in some cases provide more accurate and rich models of random-looking systems.

Based on his experimental results, Wolfram has developed the "Principle of Computational Equivalence", which asserts that almost all processes that are not obviously simple are of equivalent sophistication. From this seemingly vague single principle Wolfram draws a broad array of concrete deductions that reenforce many aspects of his theory.

Possibly the most important among these is an explanation as to why we experience randomness and complexity: often, the systems we analyze are just as sophisticated as we are. Thus, complexity is not a special quality of systems, like for instance the concept of "heat", but simply a label for all systems whose computations are sophisticated. Understanding this makes the "normal science" of the NKS paradigm possible.

At the deepest level, Wolfram believes that like many of the most important scientific ideas, the Principle allows science to be more general by pointing out new ways in which humans are not special. In recent times, it has been thought that the complexity of human intelligence makes us special -- but the Principle asserts otherwise. In a sense, much of Wolfram's ideas are based on understanding the scientific process -- including the human mind -- as operating within the same universe it studies, rather than somehow being outside of it.

Applications and Results

There are a vast number of specific results and ideas in the NKS book, but they can be organized into several themes.

One common theme of examples and applications is demonstrating how little it takes to achieve interesting behavior, and how the proper methodology can discover these cases.

First, there are perhaps several dozen cases where the NKS book introduces the simplest known system in some class that has a particular characteristic. Some examples include the first primitive recursive function that results in complexity, the smallest universal Turing Machine, and the shortest axiom for propositional calculus. In a similar vein, Wolfram also demonstrates a large number of minimal examples of how simple programs exhibit phenomena like phase transitions, conserved quantities and continuum behavior and thermodynamics that are familiar from traditional science. Simple computational models of natural systems like shell growth, fluid turbulence, and phyllotaxis are a final category of applications that fall in this theme.

Another common theme is taking facts about the computational universe as a whole and using them to reason about fields in a holistic way. For instance, Wolfram discusses how facts about the computational universe inform evolutionary theory, SETI, free will, computational complexity theory, and philosophical fields like ontology, epistemology, and even postmodernism.

The book also contains a vast number of individual results -- both experimental and analytic -- about what a particular automaton computes, or what its characteristics are using some methods of analysis.

Reception

NKS received unprecedented media publicity for a scientific book, generating scores of articles in places like The New York Times, Newsweek, Wired, and The Economist. It was a best-seller and won numerous awards.

NKS was reviewed in a large range of scientific journals. Several themes emerged. On the positive, almost all reviewers enjoyed the high-quality of the book's production, and the clear way Wolfram presented many ideas. Many reviewers, even those who engaged in other criticisms, found aspects of the book to be interesting and thought-provoking. On the negative, many reviewers criticized Wolfram for his lack of modesty, lack of mathematical rigor, and the lack of immediate utility of his ideas. Concerning the ultimate importance of the book, a common attitude was that of either scepticism or "wait and see". Detailed criticisms are outlined below.

Many reviewers and the media focused on the use of simple programs, and cellular automata in particular, to model nature -- rather than the more fundamental idea of systematically exploring the universe of simple programs.

Criticism of NKS

The book has attracted several types of criticism.

Scientific philosophy

A key tenet of NKS is that the simpler the system, the more likely a version of it will recur in a wide variety of more complicated contexts. Therefore, NKS argues that systematically exploring the space of simple programs will lead to a base of reusable knowledge.

However, many scientists believe that of all possible parameters, only some actually occur in the universe. That, for instance, of all possible variations of an equation, most will be essentially meaningless. NKS has also been criticized for asserting that the behavior of simple systems is somehow representative of all systems.

Methodology

A common criticism of NKS is that it does not follow established scientific methodology. NKS does not establish rigorous mathematical definitions, nor does it attempt to prove theorems.

Along these lines, NKS has also been criticized for being heavily visual, with much information conveyed by pictures that do not have formal meaning.

It has also been criticized for not using modern research in the field of complexity, particularly the works that have studied complexity from a rigorous mathematical perspective.

Utility

NKS has been criticized for not providing specific breakthroughs that would be immediately applicable to ongoing scientific research.

There has also been criticism, implicit and explicit, that the study of simple programs has little connection to the physical universe, and hence is of limited value.

Specific Ideas

Principle of Computational Equivalence

The PCE has been criticized for being vague, unmathematical, and for not making directly verifiable predictions. It has also been criticized for being contrary to the spirit of research in mathematical logic and computational complexity theory, which seek to make fine-grained distinctions between levels of computational sophistication.

The Fundamental Theory

Wolfram's speculations of a direction towards a fundamental theory of physics have been criticized as vague. It has also been claimed that his model is ruled out by Bell's theorem.

Natural Selection

Wolfram's claim that natural selection is not the fundamental cause of complexity in biology has caused some to state that Wolfram does not understand the theory of evolution. A common sentiment is that NKS may explain features like the forms of organisms, but does not explain their functional complexity.

Originality and Self-Image

NKS has been heavily criticized as not being original or important enough to justify its title and claims.

Edward Fredkin and Konrad Zuse pioneered the idea of a computable universe, and specifically the idea of the universe as a cellular automaton. It has been claimed that NKS tries to take these ideas as its own. Juergen Schmidhuber has been a particularly vocal critic in this regard, throwing in the additional charge that his work on Turing machine-computable physics was stolen without attribution.

It has been claimed that the core idea that very simple rules often generate great complexity is already an understood and established idea in science-particularly in chaos and complexity research.

The authoritative manner in which NKS presents a vast number of examples and arguments has been criticized as leading the reader to believe that each of these ideas was original to Wolfram.

Some have argued that the use of computer simulation is ubiquitous, and instead of starting a paradigm shift NKS just adds justification to a paradigm shift that has already occurred.

Response to Criticism

Scientific Philosophy

Wolfram argues that a typical computational system is significantly richer than a typical equation, and therefore exhaustively enumerating computational systems is more likely to result in diverse and novel behavior. In fact, Wolfram argues that this is a direct result of how science has evolved to minimize complexity and computation.

In the history of science and mathematics, simple ideas tend to be reused much more often than complicated ones - if not immediately, then decades or sometimes even centuries later.

Methodology

A major claim of NKS is that mathematical descriptions of computational systems are incapable of capturing their essence. Therefore it is no surprise that NKS doesn't try to employ mathematical descriptions.

Wolfram claims that just as you don't need a formal definition of life to do biology, you don't need a formal definition of complexity to do NKS.

Because mathematics cannot reduce the complexity of simple programs, Wolfram argues that visualization is the first and most important step. One does not, for instance, try to figure out what a CPU is doing by taking its temperature. If the system is simple, the visualization is direct and meaningful.

Utility

The NKS book itself applies Wolfram's ideas to a vast range of issues, challenging the claim that they cannot be easily applied. However, they are often applied from a non-traditional perspective; and it is understandable that the questions that people ask within one paradigm are not the same questions that can be asked within another paradigm.

If there are universal laws that govern the behavior of systems -- or universal patterns that emerge across a broad range of systems -- then these will govern/occur in simple computer programs as well as in the natural world. The only way to deny this is to argue how the rules of the universe are different from abstract rules in general.

Specific Ideas

Principle of Computational Equivalence

Like many principles in science, the PCE cannot be axiomatically derived from existing knowledge. For instance, a similar statement is the Church-Turing thesis, which cannot be proven yet serves as a guide to intuition and understanding in computer science.

Originality and Self-Image

Much of this criticism stems for a misidentification of the core thesis of NKS. NKS is not about the fundamental theory of physics, or if the universe is discrete and computable.

NKS is about systematically and experimentally exploring the universe of very simple programs, and also it's about using very simple programs to model aspects of nature. These issues are independent of whether the ultimate model of physics is itself a simple program.

When the book was published in 2002, very little if any scientific research was based on using very simple programs in those two ways.

See also

External links

Official site

Wolfram, Stephen, A New Kind of Science. Wolfram Media, Inc., May 14, 2002. ISBN 1579550088
Wolfram, Stephen, "Quick takes on some ideas and discoveries in A New Kind of Science". Wolfram Media, Inc.
NKS 2004 conference. Wolfram Media, Inc.
InformationSpace. Causal set exploration tool which supports 1 dimensional causal sets such as those found in the book.

Reviews and Overviews

Clark, Ed, Reviews (the most comprehensive collection of reviews of NKS)
Amazon.com, book reviews (ranked by votes)
Schmidhuber, Juergen "A 35 year old kind of science" (with links to much earlier work on digital physics)
Boguta, Kovas, "Comments on a review of NKS" (an exposition of what NKS actually is, framed as a response to critics)
Review and discussion of A New Kind of Science. Slashdot.
Krantz, Steven G., "Book review". American Mathematical Society (PDF document)
Rucker, Rudy, "Book Review". American Mathematical Monthly, November, 2003.
Stephen Wolfram's lecture at MIT, "A New Kind of Science" (Real Media video and audio, 1:36:36). Department of Mathematics at MIT, September 15, 2003.
Naiditch, David, "Divine Secrets Of the Ya-Ya Universe. Stephen Wolfram: A New Kind of Science — or a Not-So-New Kind of Computer Program?", Skeptic Magazine, issue 10-2, 2003.nl:Computational equivalence