Lies, damned lies, and statistics 

Dishonesty is popularly classified under the headings: lies, damned lies, and statistics. 

Political axes are ground with statistics. Statistical studies are used as political ammunition in everything from gun control to smoking bans to nuclear power. Remember the study which drew scary conclusions about diet soda from the fate of mice fed massive doses of saccharin? If you don't like the result of one statistical study, you need only wait until your side comes up with a study of its own! 

If these shenanigans have left you with a profound distrust of statistics, it turns out that you are right! The root of the problem is that orthodox statistical theory is wrong in principle! Says who? Says Edwin T. Jaynes, professor of physics at Washington University of St Louis. His book--Probability Theory: The Logic of Science--is to be published by Cambridge University Press. It presents the results of decades of work--a highly developed mathematical theory which changes, well, everything! 

Bayes' theorem 

Jaynes has long been a prominent advocate of "Bayesian" methods in statistics, i.e., methods based on the use of Bayes' theorem. Bayes' theorem tells you how to update a probability on the basis of new evidence. Orthodoxy rejects it. 

Bayes' theorem is deduced from probability theory, so when orthodoxy rejects Bayes' theorem, it implicitly rejects probability theory. Instead of using probability theory, orthodoxy resorts to a hodgepodge of sneaky dodges, "adhockeries." 

So what? So orthodoxy loses! Jaynes presents proof that probability theory is dictated by logic. If you assign numerical measures of plausibility to propositions, and wish to reason consistently about them, then you must use probability theory. 

This is no nit-picking dispute between rival academics. Nothing less is at stake than the nature of the universe! 

Random universe? 

Orthodoxy is "frequentist." It holds that probability and statistics deal only with random events, and that probabilities are merely the long run average frequencies which would be found in repeated random trials. On this view, probability is intrinsic, a fact which owes nothing to anyone's knowledge or ignorance. 

This explains the orthodox rejection of probability theory. Probability theory tells you, via Bayes' theorem, that you must update a probability to reflect new evidence! But an intrinsic probability could hardly be changed by a mere change in someone's knowledge. An intrinsic probability would be whatever it was no matter who knew what, like the distance to the Sun. 

Intrinsic probabilities require, suppose, and depend upon intrinsic randomness. And what, pray tell, is intrinsic randomness? It is the property some things allegedly have of "just happening," without causes. But intrinsic randomness is not to be found: causality is universal. 

The new probability theory does not need intrinsic randomness--indeed it has no room for it! 

Card trick 

An example will illustrate the issues. 

I shuffle a deck of cards, and place it face down on a table. This is the kind of example beloved of orthodoxy, but nothing in it is intrinsically random. The details of the shuffle were what they were, and led to the order of cards to which they in fact led. Since there is no randomness here, supporters of orthodoxy can't logically get started. (Of course, they don't let that stop them in practice!) 

Now I ask you what is the probability that the top card is the ace of spades. If you've had some background in gambling or probability, you will say 1/52. This is your prior probability, i.e., prior to the new evidence I'm about to give you. 

I lift the top card, you see which card it is, and I replace it face down on top of the deck. Again I ask what is the probability that the top card is the ace of spades. Your answer this time will be very different--either zero (if the top card isn't the ace) or one (if it is the ace). The probability has changed! 

So what? So the only thing that can account for that change is your knowledge of the top card's identity. Everything else is the same: same cards, same shuffle, same order, same everything. In all the universe, the only relevant change is in your knowledge--yet that changes the numerical probability you assign. A probability is a measure of a state of knowledge. 

You could display this example as an application of Bayes' theorem--if you wanted to crack a peanut with a sledgehammer. 

Prior probability and prior knowledge 

But where did you get that 1/52, your prior probability? Orthodoxy claims that you had to dream it up. They claim that prior probabilities are subjective. 

Not so. Prior probabilities can be calculated objectively on the basis of prior knowledge. Much of Jaynes book is devoted to deriving methods of doing just that. 

In the card example, you started by knowing that there are 52 different cards, and knowing that you did not know any differences among them which would affect the shuffle. It can be shown that this prior knowledge logically obliges you to assign a prior probability of 1/52. 

Few problems are as simple as a deck of cards, but the same kind of reasoning has been generalized to yield potent methods for calculating probabilities from prior knowledge. Much of Jaynes' book is devoted to deriving and explaining these methods--and applying them to everything from statistical mechanics to the kinds of fish in a lake. They remain a topic of active research. 

These methods use what you do know, while avoiding assumptions about what you do not know. The exclusion of such assumptions accounts for everything which orthodoxy attributes to randomness. 

Extended logic 

The alleged weakness of Bayesian methods, reliance on prior probabilities, can now be seen as a great strength. Prior probability turns out to mean prior knowledge, which can be used systematically to improve your calculations. 

Orthodox statistics, on the other hand, has no formal way to use prior knowledge in its calculations! The theory simply has no place to put it! Orthodoxy must approach each problem with an assumption of randomness, but without specific prior knowledge--i.e., with all the cunning and shrewdness of a new-born puppy! 

With prior probabilities based on prior knowledge, probability theory becomes an extension of logic. The prior knowledge serves as its premises. When your prior knowledge lets you draw conclusions with certainty, probability theory reduces to ordinary logic. 

Probability's practicalities 

Probability as logic has great practical advantages. 

Many calculations which take 2 or 3 lines using probability as logic would require pages with an orthodox approach. Probability as logic can be applied to any problem where the evidence is fragmentary--not merely to those which you can imagine to involve randomness. In many problems, it can be proved that probability as logic gives the best possible answer from the data you have--in some cases so much better than conventional results as to strain belief! 

Jaynes tells of a computer program using probability as logic which gave results many factors of 10 better than the conventional approach. This was announced, but it wasn't believed! It was regarded as much too good to be true! To save their credibility, Jaynes and Bretthorst (the programmer) were obliged to give away copies of the program and of sample data sets so doubters could see for themselves! 

Probable revolution 

Quantum mechanics is central to physics and chemistry, which in turn are central to virtually all of science. Orthodox quantum mechanics is through and through probabilistic, and has a decided anti-causal bias. (So much so that jokes are made about "quantum theology.") Probability as logic changes the meaning which may validly be attached to all of quantum mechanics! 

Expect a scientific revolution as the new theory spreads. 

Risky stuff 

Much bogus fear-mongering is about to become obsolete. 

Consider the problem of calculating toxicity. Conventionally, it is assumed that the harm due to kilodoses of whatever (remember the mice and their saccharin?) can be simply extrapolated back to low doses. Jaynes points out that this leads to gross over-estimates of the risks of low doses. It flouts prior knowledge that most substances are rapidly eliminated from the body. Orthodox theory cannot use this prior knowledge. Probability as logic can use it. 

More generally, risk is a matter of probability. Those who use risk as a pretext for statist measures will no longer be able to present estimates of risk as intrinsic, "just the way it is" revelations. They will be obliged to present the prior knowledge (or error!) on which their estimates are based. Those with different prior knowledge can validly present different estimates. 

Pregnant thoughts 

We can now recognize probability theory as a quantitative branch of epistemology, the part of philosophy which deals with the nature and means of knowledge. 

To project the stupendous consequences of a quantitative epistemology, compare it with the revolution that hit physics when Galileo quantified motion. Or the revolution that hit chemistry when Lavoisier insisted on precise measurement. Then recognize that physics and chemistry are small potatoes compared to epistemology, which deals with all knowledge! It will change, well, everything! 

It's early days, but the path of advance is marked out. It will be traveled on a time-scale of centuries! 

Spread the word! 

Don't expect the establishment press to trumpet these discoveries! Probability as logic is profoundly out of step with establishment thinking. If people are to hear of it, you will have to tell them. This is a job for quackgrass activists! 

Why bother? A true, valid fundamental theory conserves the most precious of resources: the time and mind-power of human beings. 

Theory is a thinker's prime tool. The better his theory, the better his thinking, the fewer his mistakes, and the better his chance of hitting on important new discoveries. If you pass along this new tool, it will eventually reach those in every field who are creating the new renaissance--and multiply their power. 

Read the book! 

Screw up your courage to the sticking place, and read Jaynes! The math is a bit hairy for most of us, but there is plenty of nice, clear text that is loaded with good ideas. (The above link will lead you to a page with a raft of links on probability as logic, including a link to Jaynes' book and assorted papers by Jaynes and others.)

• Jaynes' book is available in PDF format here, where you can download it chapter by chapter.To read PDF, you need Acrobat Reader. If you don't yet have Acrobat, get it FREE here; it's worth having for general purposes.
• If you have the GnuZip compression utility, this link will get you a compressed copy of Jaynes' entire book in PDF format. About 4 MB.
• Larry Bretthorst has a related book online in PDF: "Bayesian Spectrum Analysis and Parameter Estimation" About 1.3 MB.

---

Note:

Robert Matthews published an excellent report in The Sunday Telegraph on the connection between non-Bayesian statistics and today's flaky science—the bogus health scares and the miracle cures that don't work.

Matthews focuses especially on Ronald Aylmer Fisher's P-value, and its arbitrary significance level of P<0.05, but notes that "alternative [orthodox] methods are know to suffer similar flaws to P-values, exaggerating both the size of implausible effects and their significance."

Matthews' The Great Health Hoax is online at the Science and Environmental Policy Project website

Michael Miller is an engineer and Objectivist filosofer with thirty years of experience. He had been a member of Boycott Alberta Medicare in 1969 and of the Association to Defend Property Rights from 1973 on. He writes in-depth philosophical theory at his publication, Quackgrass Press, which can be accessed at http://www.quackgrass.com.