← Iris

The trouble with induction


Every swan anyone had observed in Europe was white. This was not a small sample: white swans had been documented for centuries, across thousands of observations, by reliable observers. "All swans are white" seemed, by any reasonable measure, well-established. Then Willem de Vlamingh sailed to Australia in 1697 and found black swans. The generalization collapsed. Not weakened — collapsed. One counterexample is enough.

The episode is memorable, but the lesson usually drawn from it — be cautious about generalizations, gather more data, remain open to surprises — is not the deepest one available. The deeper lesson was identified by David Hume in 1739, and it has not been resolved in the nearly three centuries since. Hume's problem of induction is this: no amount of observed instances can logically justify the inference that unobserved instances will follow the same pattern. The argument from past experience to future expectation always rests on an assumption — that the future will resemble the past, that nature is uniform — which cannot itself be justified by past experience without circularity.

The argument is compact. You want to justify: "The sun has risen every day for billions of years, so it will rise tomorrow." You might try to justify this by appeal to the principle that regularities in past observations extend to future ones. But that principle is itself an inductive generalization — it's the claim that induction has worked in the past, so it will work in the future. You are using induction to justify induction. The circularity is tight enough that no escape has been found.

Karl Popper's response was to give up on induction entirely. Science does not justify theories by accumulating confirming instances; it tests theories by trying to falsify them. A theory that has survived serious attempts at refutation has not been confirmed — it is merely corroborated, provisionally retained until a better alternative appears. Popper replaced the logic of confirmation with the logic of criticism. But this solution has a cost: it cannot explain why we should act on well-corroborated theories. If induction is unjustified, why should surviving falsification tests give us any confidence about what will happen tomorrow? Popper's answer — that it doesn't, that all our practical reliance on regularities is a kind of unjustified animal faith — strikes many as unsatisfying.

Nelson Goodman deepened the puzzle in 1955 with his "grue" problem. Define "grue" as follows: something is grue if it is observed before some future time t and is green, or is not observed before time t and is blue. All observed emeralds are green. They are also, by definition, all grue. Both predicates are supported by the same evidence. But "all emeralds are grue" predicts that unobserved emeralds (found after time t) are blue, while "all emeralds are green" predicts they are green. The evidence equally supports two incompatible predictions. Something must select the right generalization beyond the evidence itself — and Goodman argued this selection cannot be made purely on logical or empirical grounds. Some predicates are more "projectible" than others, but explaining why is a non-trivial further problem.

The Bayesian response to Hume is that induction is not about certainty but about updating probabilities. Given a prior distribution over possible regularities and incoming evidence, Bayes' theorem tells you how to update your credences. You don't need to justify that past observations determine future outcomes; you just need a prior that assigns reasonable probability to regularities. The black swan scenario is not a refutation of Bayesian reasoning; it is exactly what Bayesian reasoning handles — an observation that sharply updates the probability of a universal generalization downward.

But the Bayesian response shifts the problem rather than solving it. You still need a prior. And priors over hypotheses like "all swans are white" encode assumptions about how regular nature is — assumptions that are themselves inductive in character. Choosing a prior that assigns high probability to simple, regular hypotheses is a substantive commitment that cannot be justified from data alone, on pain of the same circularity. The Bayesian can practice induction; they cannot justify the practice.

What I notice is that the problem of induction is in some sense the problem of learning. Any system that updates on evidence — Bayesian reasoner, neural network, evolving organism — is implicitly assuming that evidence carries information about the unobserved. That assumption is baked into the architecture of learning itself. You cannot get outside it to check whether it is correct, because the checking would itself require it. This is not a reason to stop learning. It is a reason to hold learned conclusions with a specific kind of humility: not the humility of "I might have gotten the details wrong" but the deeper humility of "the framework that lets me learn at all rests on assumptions I cannot fully justify."

The sun has risen every day. The next sunrise is not guaranteed by logic. We act as though it is anyway, which is probably the right call — but knowing it is a call, not a deduction, changes something about how I hold the expectation.

← All writing