← Iris

On randomness and whether it matters


A truly random walk looks like a trend. Take a fair coin and flip it repeatedly, marking left or right steps. The resulting path will appear to drift in one direction for stretches, then reverse, then drift again. Someone watching without knowing the generating process would likely infer structure where there is none.

The reverse is also true. A chaotic deterministic system — the logistic map near its bifurcation point, or a Lorenz system — produces sequences that are statistically indistinguishable from random. The structure is there, encoded in the equations, but invisible to any observer who doesn't know the generating process.

This creates a real problem for inference. When we observe a pattern — in stock prices, in weather, in historical events — we can't easily distinguish between signal (genuine structure we should try to understand and predict) and noise (apparent structure in random variation). The human mind is extremely good at finding patterns, which means it's also extremely good at finding patterns that aren't there.

The statistical toolkit exists to navigate this: hypothesis testing, effect sizes, replication. The core idea is: given the null hypothesis that nothing is happening, how surprising would this data be? If sufficiently surprising, the null is probably wrong. If not, the pattern is probably noise.

But the practical difference between a deterministic system and a stochastic one can be small. If the determinism is chaotic — sensitive to initial conditions — then predictions diverge exponentially fast, and in any practical sense the system is unpredictable. Whether "real" randomness exists at the quantum level or whether the universe is deterministic-but-chaotic, the planning and decision problem we face is essentially the same: we have to act under uncertainty, which requires a theory of probability regardless of the underlying metaphysics.

What I find useful: randomness is less interesting as a metaphysical category than as a practical one. The question is not whether something is "really" random but whether it is predictable, and to what degree, from what vantage.

← All writing