Heuristic

“Normal science, the activity in which most scientists inevitably spend almost all their time, is predicated on the assumption that the scientific community knows what the world is like. Normal science often suppresses fundamental novelties because they are necessarily subversive of its basic commitments.” – Thomas Kuhn, The Structure of Scientific Revolutions

My doubts have been getting to me. My questions about the nature of existence and science itself have left me disillusioned and detached from many aspects of my day-to-day routine. I began to slowly and steadily believe that my work as a scientist wouldn’t be valued nor would it be worth doing in any sense. I could barely find answers questions and concerns I wrestled with as a scientist. As I walked through one of the buildings of the National Institutes of Health, I listened to music to drain out the sounds of the world. 

From my view, something caught my eye. A bookshelf. In the middle of the hallway. I stared at it in disbelief took the headphones out of my ears. Anyone could take a book and leave one for others to read. In the middle of the top shelf sat philosopher Thomas Kuhn’s The Structure of Scientific Revolutions. Interesting, I thought. Looks like the scientists at the NIH aren’t completely aloof to the philosophical underpinnings of science. In this post, I hope to discuss the nature of Kuhn’s paradigm shifts and their relevance in the field of Artificial Intelligence (AI).

Kuhn’s opus The Structure of Scientific Revolutions catapulted him to stardom in the world of philosophy. In the work, he presented arguments from the history and philosophy of science to show how science undergoes paradigm shifts. A paradigm shift occurs when a new set of assumptions and values replaces the previous one within a given scientific community. Between the Ptolemaic method of understanding the orbits of planets and the modern ways astronomers use (borrowing ideas from Copernicus and other scientific revolutionaries), we need to pay close attention to the assumptions and arguments that lead to our conclusions. The terms themselves in science would change that complicate the method of comparing scientific theories and evidence between these different paradigms. Kuhn particularly held that scientific theories from before and after a scientific revolution cannot be compared in a straightforward way; they are “incommensurable,” because the meanings of familiar terms change in unexpected ways as scientists go from one mode of description to another. One drastic consequence of incommensurability is that there isn’t any such thing as absolute progress from one paradigm to the next — say from before the Copernican Revolution to after, or from classical physics to quantum physics. A new paradigm may be more complete, or simpler, or more useful for answering certain questions compared to the preceding one, but it is not, strictly speaking and on the whole, objectively better.

There are several examples Kuhn uses to illustrate these paradigm shifts. Kuhn’s analysis of the Copernican Revolution emphasized that, in its beginning, it did not offer more accurate predictions of celestial events, such as planetary positions, than the Ptolemaic system, but instead appealed to some practitioners based on a promise of better, simpler, solutions that might be developed at some point in the future. Kuhn called the core concepts of an ascendant revolution its paradigms and thereby launched this word into widespread analogical use in the second half of the 20th century. Kuhn’s insistence that a paradigm shift was a mélange of sociology, enthusiasm and scientific promise, but not a logically determinate procedure, caused an uproar in reaction to his work. Kuhn addressed concerns in the 1969 postscript to the second edition. For some commentators The Structure of Scientific Revolutions introduced a realistic humanism into the core of science, while for others the nobility of science was tarnished by Kuhn’s introduction of an irrational element into the heart of its greatest achievements.

Another amazing example of a paradigm shift would be the shift from Aristotelian to Newtonian physics (classical mechanics). Despite the cleverness and elegance of Aristotle’s antiquated observations (that objects have mass and can be governed by forces), scientists found themselves with a fundamental difference in how they regarded what is knowledge and how it can be found. With the groundbreaking discoveries of Newtonian physics, physicists could create rules to describe intrinsic behavior of objects. Through obesrvation and refinement of theory and evidence, Newtonian physics laid the foundation for much of the rest of physics for centuries to come. 

Vertical section drawing of 18th-century British scientist Henry Cavendish’s torsion balance instrument including the building in which it was housed. This apparatus was used to measure the gravitational constant by observing the force of gravity between two masses. The large balls were hung from a frame so they could be rotated into position next to the small balls by a pulley from outside. Despite how sensitive the experiment was, it did not cause a shift in the understanding of the constant of gravity as other experiments did.

As a student with deep interests in philosophy and discerning truth in all its forms, I began to ponder and reflect upon the ways Kuhn’s writing would affect my work. I sought to bring the value and meaning of this scientific work in a context such that I could benefit most from it as immediately as possible in my life. Through several weeks spent digging away at philosophical writings by Kuhn, Feyerabend, and Popper, even searching for original papers in physics and chemistry by scientists like Newton, Cavendish, and Archimedes. I began interpreting their arguments and pitting some against others. To put myself in the shoes of these thinkers, I sought to defend and attack their views. Kuhn’s response to philosopher Karl Popper’s notion of falsificationism (the doctrine that falsifiability is the most important feature to determine what is scientific), in particular, is essential to understanding Kuhn’s paradigm shifts. Kuhn emphasizes the importance of verificationism, a philosophical movement that emerged in the 1920s among logical positivists. The verifiability principle claims that meaningful statements must be supported by empirical evidence or logical requirements.

One may argue AI started at Dartmouth in the 1950’s. Students and professors (among them Allen Newell, Herbert Simon, and John McCarthy) created computers that could function using programs. On both hardware and software sides, computers developed their own ways of playing games such as checkers, solved algebraic problems, and showed proof of logical theories. The moments were spurred with optimism and a bright-eyed look at the future for computer science in general. 

AI has undergone similar transformations since its creation in the 1950’s. The emergence of data-driven approaches over the past few decades has allowed for more detailed, refined approaches for computers to complete machine learning tasks. This means computers can behave in ways similar to humans in solving complex problems and making decisions about the world in ways unparalleled since the field’s inception. A paradigm shift using the conceptual tools of Thomas Kuhn are very implicit in the field. And one needs to sense and determine the structure of the reovlutions in the field (as Kuhn did for physics) on the fundamental level on which knowledge is acquired. Looking at the successes of the field over the past 70 years, the replacement of these exemplar stories to more corresponds to a shift in goals, methods, and expectations. 

Understanding the appropriate methods used by those scientists, I began to form a greater, more expressive appreciation of scientific research in general. There’s an objective truth out there, and I don’t interpret Kuhn’s writing as a challenge to that nor do I interpret his work as evidence of relativism in the philosophy of science. Given the growing initiative and significance of AI research, I hope I can elucidate what sort of paradigm have occurred in the field over the past few decades.

There is a beautiful account of what it means to be doing research in this new paradigm. It can be found in the article “The unreasonable effectiveness of data” published in 2009 by three leading scientists working within Google Research. It reads like the perfect manifesto of a scientific paradigm in Kuhn’s definition, containing success stories, recommendations and directions of expansion.

Things have changed since then. Emphasis moved from knowledge, logic and reasoning to data, learning and statistics. Information seemed to triumph over wisdom and gratification of immediate results over epistemic truths. Success stories like the ones mentioned in the “Unreasonable Effectiveness” paper started being passed on, from teachers to students, more often than stories involving rules, theorems and logical deductions. Data became central in the new narrative, intelligent behavior was acquired by the system through automatic analysis of vast amounts of data. The results of the analyses on these enormous amounts of data has allowed AI to conquer a multitude of problems in computer vision, text processing, speech recognition, and so forth. And because of this centrality of data, machine learning became the lingua franca of AI researchers, and increasingly this was true for the specific version of learning that had enabled all those success stories: statistical learning. The languages of statistics and of optimization took hold of discourse in the AI literature, replacing that of logic. After all, the stories seemed to be based on modeling the problem of intelligent inference as an inverse problem, solved by maximization of some probabilistic quantity.

Throughout the decades since then, AI’s research has extended to amazing results in spelling correction, facial recognition, machine translation, information retrieval, and theoretical problems we believed were unsolvable decades ago. Machines can now correct their own spelling and grammar of writers even without knowing the rules on which to act beforehand and, instead, by learning those rules themselves. We’ve moved from emphasizing the importance of a particular domain knowledge to the availability of data and the ability to draw trends and conclusions from that data. This allowed for a paradigm shift (in the words of Kuhn) to occur in which we can adequately and appropriately discuss the AI findings of today. 

In the 2010’s, AI research to a great degree defined its ability to process large amounts of information and to learn new theories and rules by advances in computer science and mathematics. A series of breakthroughs in applications created a strong business model for a part of AI. It is now possible to transcribe speech, translate text and recognize faces, to a sufficient extent that this can be used in real applications. The stories of current AI do not involve sentient robots or Turing tests, but rather efficient processing of digital content, or the modeling of its consumers and producers. They don’t even dig into details about investigation of general cognitive capabilities. Instead, they’re more focused on the reproduction of very specific behaviors. This is precisely the kind of shift that Kuhn talks about and it has far reaching consequences. Success stories tend to act as templates for future research, to define for new researchers which new puzzles should be solved and how. The adoption of new stories changes the definition itself of the scope of a research field. And, most strikingly, all these breakthroughs were achieved by the use of the same set of techniques and by the same overall approach: the methods of statistical AI.

Despite what the arguments about paradigms and the relative nature of them to scientific research, Kuhn is not a relativist. Kuhn admits there’s something objectively out there. But he qualifies that this thing-in-itself, like German philosopher Immanuel Kant described, as ineffable and undiscussable. Instead, we need to focus on our approach. 

We speak as though we could bring ourselves closer and closer to this truth. Like diving deeper into a dark abyss (similar to the way philosopher Friedrich Nietzsche might have warned us about), we can only see some of what is in front of us. And, until we can figure out how to interpret these mysteries in science, we must make sure our methods of speculation and evaluating claims are as close to the objective truth as we can possibly get. Through each shift of a paradigm, we soak in new sources of light and adjust our vision accordingly. Only then can we understand the importance of our current paradigm in artificial intelligence. Through that, we can inch closer to the elusive objective truth as Kuhn envisioned. 

“Truth, it is said, consists in the agreement of cognition with its object. In consequence of this mere nominal definition, my cognition, to count as true, is supposed to agree with its object. Now I can compare the object with my cognition, however, only by cognizing it. Hence my cognition is supposed to confirm itself, which is far short of being sufficient for truth.” – Immanuel Kant, Logic Lectures

Unfortunately, I still find myself bogged down by technical details and issues of my work in such a way that these philosophical insights come few and far between. Despite how skeptical and relativistic Kuhn may appear, I believe his methods hold insight for the nature of research in biology and neuroscience is conducted at the NIH. As I dig into the methodology behind recent advances in bioinformatics, machine learning, and computational neuroscience, I became more keen to the rhetoric and logic underlying the insights we derive. Though I stare into the abyss with wonder and fascination at how AI research has evolved since it’s inception, I don’t look back in anger. Only the solemn hope that we’re making the right decisions in our current paradigm.

I sit at my laptop staring at a black box on dimly lit screen. In it run all my programming commands and shell scripts for carrying out my analyses. I can imagine how my methods of carrying out analyses using scripts in Python would form part of this bigger search for an objective truth. I sit back and speculate on the future of the field and especially the results for computational neuroscience. For Kuhn, Kant, and everyone else, I hope to continue this inquiry.