The Epistemic Risk of Flawed Science

Friday 07 June 2024

In last week’s newsletter I introduced the concept of epistemic enclaves to characterize the knowledge regimes of distinct communities. The idea of epistemic enclaves follows straight-forwardly from an appreciation of the constructed character of human knowledge. Different communities construct their knowledge in different ways, and so find themselves with distinct but overlapping epistemic regimes. Since scientific knowledge is the most objective form of knowledge that human beings have formulated, we hesitate to see the scientific community as a community with its own distinctive knowledge regime, but we need to see this as a matter of degree. Scientific knowledge is certainly more objective than most folk knowledge regimes, but it isn’t absolutely objective. Even within science, as is widely recognized, there are disciplines that we characterize as “hard” sciences that seem to possess a greater degree of objectivity, and “soft” sciences with a lower degree of objectivity.

Scientists have done their utmost since the scientific revolution to distance themselves from the philosophy that was the womb in which science was conceived and brought to term. As part of this campaign for the independence of the epistemic enclave of science, Plato has been a particular target, and his otherworldly philosophy has been seen as antithetical to scientific knowledge. Nevertheless, the concept of knowledge implicitly held by many scientists is essentially Platonic. I am going to call the Platonic conception of knowledge that conception holding that (true, genuine) knowledge is eternal and unchanging. Scientists aren’t likely to use these descriptors, but presuppositions have a way of transcending language.

The long and difficult road of research can still ultimately open out onto a Platonic conception of knowledge.

One can make any number of compromises between naturalism and the Platonic conception of knowledge that will allow one to conceal (from oneself, if not also from others) the Platonic character of knowledge that is being presupposed. For example, one might hold that the process of scientific discovery is a long, uncertain, and rocky road, with many setbacks, but the implication is that it does eventually lead to genuine knowledge (also known as “settled science”). I could call this the teleological cope: scientific knowledge ultimately converges on eternal and unchanging knowledge (i.e., the Platonic conception of knowledge), even if this convergence is inevitably and perhaps also indefinitely delayed. Thus one is not claiming that current scientific concepts and theories are timeless and unchanging, but they are converging on timeless and unchanging knowledge. The second part is the quiet part that you’re not supposed to say aloud. The teleological cope is a concession to the contingency that is central to naturalism, but it is only a limited concession, and it only satisfies an inquiry that is willing to be satisfied by half measures.

In the previous newsletter I said that epistemic enclaves are not only about bare facts, but as much about facts that are bearers of meanings and values. This enters quite directly into history, and can lead to radically different accounts of the same event or set of events. (The long-debated scientificity of history provides us with a test case of a knowledge regime that can make claims to objectivity, but which is all-too-often subjected to pressures that deviate from ideal scientific standards of objectivity.) All history must select the evidence from which it draws to construct its narrative. The principle of the selection of relevant evidence is, of course, the Achilles’ heel for objectivity. Since we must select evidence, we select on the basis of the value of that evidence to a historical account, and this is an axiological judgment, and not a judgment about the legitimacy of the evidence or the objectivity of the facts concerned. It is entirely possible, even if not ever realized in fact, that two historians (or any two individuals giving an account of some episode) might weigh the importance of facts so differently that they come away from one and the same milieu with two disjoint sets of facts, and each constructs a narrative of the episode in which there are no facts in common.

Multiple observers of one and the same event could give disjoint accounts of what happened if they value evidence differently, which is why we often find that eyewitnesses differ in their testimony.

This is, or would be, a particularly radical outcome. As I said above, this might not ever happen in fact, but we cannot exclude the possibility, and what is true of history is true of other sciences, including the hard sciences. The sciences select the evidence they will use in formulating a theory, which is the point of the anecdote related by Eugene Wigner as quoted in newsletter 282. Part of the value of a scientific research program is that it aligns the scientific community (or the greater part of the scientific community), and this alignment extends to what will be considered relevant evidence and what will not be considered relevant evidence. There are other alignments that also serve to constrain what we value as relevant evidence. As human beings, we have human experience in common, and we might call this the human alignment, which consists of a complex network of cognitive biases and shared evolutionary psychology that tends to unconsciously coordinate our experience, making it possible to agree upon much that might otherwise be elusive.

The ultimate alignment would be a science of science that entailed precise quantitative and axiological constraints for evidence. If there were a science of science, then we might very well argue that science is converging on absolute knowledge (without this being a teleological cope), that science can be “settled” once an area of knowledge has been exhausted, that the growth of scientific knowledge can be programmatic and therefore automated, and so on. We cannot rule out the automated pursuit of science, but science at its present state of conceptual development means that automated science would not be absolute knowledge or ultimate science. The kind of science we might program a machine to pursue wouldn’t be a perfect or universal science, but only one more permutation of the possible pathways for science, as though machines already constituted their own machine civilization and that machine civilization was another cultural-historical type to add to Danilevsky’s list of cultural-historical types.

Can science be automated? Perhaps, but if a flawed science is automated, it will not be improved by the automation, only amplified. Garbage in, garbage out.

Undertaking science in a framework defined by personalities, blindspots, idiosyncratic insight, and adherence to a political narrative entails what we can call epistemic risk. Epistemic risk can take the form of falling afoul of a powerful personality (or being too close to them and losing one’s autonomy), failing to see relevant evidence due to a blindspot (or becoming aware of the blindspots of others, and thus falling afoul of the research community), failing to possess the requisite insight to make a scientific discovery (or having an unwelcome insight that runs counter to the prevailing scientific research program), or being so captured by a political narrative that knowledge growth stagnates (or running afoul a political narrative by attempting to pursue a scientific research program counter to the narrative). Epistemic risk only enters into science because there is no science of science. If scientific discovery were an exhaustively theorized activity, the continuation of which can be generated by an algorithm, there would be no epistemic risk involved.

It remains to be said that, even if we possessed a science of science, we could not be assured of its uniform deployment. The formulation of an adequate science of science would only present the possibility of reforming science, but not the reform itself, which latter would be an historical process distinct from the ideal end state of mature scientific knowledge attained (i.e., the convergence of scientific knowledge upon the Platonic conception of knowledge). We can easily see that this historical process of the deployment of a science of science would be riddled with dangers. A science of science, if acted upon, would collapse epistemic enclaves. Indeed, the ability of such a discipline to collapse epistemic enclaves would immediately be perceived as a threat, and those in power within given epistemic enclaves would act to counter this threat.

How much knowledge remains hidden in the blindspots of science?

In the long run, possession of a science of science would greatly expand scientific knowledge, and insofar as scientific knowledge can be the source of technologies that can be engineered into industries, a society adopting and acting upon a science of science would eventually come to dominate militarily and economically, and in the anarchic nation-state system, this kind of power is the only real currency. However, there is a great gap — perhaps even a yawning chasm — between the rudiments of a science of science being formulated, and a mature science of science deciding the fates of nation-states. This is a process that could unfold over hundreds or thousands of years — a period of time sufficient that any number of other events or historical processes might unfold, either to derail the unfolding of a science of science, to bring about an untimely end to human civilization, or to suggest alternative ideals as the proper alignment for human civilization.

In many blog posts and essays I have offered possible definitions of what would constitute a properly scientific civilization (for example, “Pathways into the Deep Future,” “The Role of Science in Enlightenment Universalism,” and “The Infinite telos of Reason: Edmund Husserl and Scientific Civilization”). Given my analysis of the institutional structure of civilization, the obvious answer to this question is that a properly scientific civilization is a civilization that has science as its central project. However, these reflections on the nature of science and the possibility of a science of science point to a further complexity.

If science in its present state of development, i.e., science that falls short of the ideal of scientific knowledge, were to be installed as the central project of a civilization, such a civilization could be as flawed in its pursuit of scientific knowledge as is Enlightenment era civilization. In such an eventuality we would experience the founder effect at the scale of civilization: the new civilization would be based only on the science that founded it, and would likely exclude a more comprehensive conception of science, perhaps on principle, as scientific variants might appear to a scientific establishment as forms of heresy. I can think of at least four obvious permutations of this problem:

1. A scientific civilization would inevitably be as flawed and as limited as any non-scientific civilization (where a non-scientific civilization is understood to be a civilization that does not have science in some form as its central project), because it would be based on flawed and imperfect science.
2. No civilization would be prepared to install science as its central project unless or until it had developed and deployed a mature science of science (scientific maturity comes first).
3. No civilization would be capable of developing and deploying a mature science of science unless and until it had installed science as its central project (a scientific central project comes first).
4. The process of science acceding to the institution of civilizational central project would be integral with and inseparable from the development of a science of science; no civilization would be capable of formulating a science of science unless it were in the process of installing science as its central project, and vice versa (scientific maturity and a scientific central project are parallel and linked).

There are probably many other possibilities, and many possible variations on these themes. The limited schematization I have employed above implies, for example, the possibilities of the development of a mature science of science without the appearance of a properly scientific civilization, the appearance of a properly scientific civilization (if it could be called such) without the development of a mature science of science, or an initial development toward both scientific maturity and a scientific central project that stagnates or is derailed before it can come to fruition.