Death is bad, 2

I fully expected the last post to be a one-shot, but then Scott Alexander wrote a thing on ethics offsets:

Some people buy voluntary carbon offsets. Suppose they worry about global warming and would feel bad taking a long unnecessary plane trip that pollutes the atmosphere. So instead of not doing it, they take the plane trip, then pay for some environmental organization to clean up an amount of carbon equal to or greater than the amount of carbon they emitted. They’re happy because they got their trip, future generations are happy because the atmosphere is cleaner, everyone wins.
We can generalize this to ethics offsets. Suppose you really want to visit an oppressive dictatorial country so you can see the beautiful tourist sights there. But you worry that by going there and spending money, you’re propping up the dictatorship. So you take your trip, but you also donate some money to opposition groups and humanitarian groups opposing the dictatorship and helping its victims, at an amount such that you are confident that the oppressed people of the country would prefer you take both actions (visit + donate) than that you take neither action.

 The concept is probably unappealing to a certain sort of person, but not me. My sort-of-utilitarian, definitely-consequentialist mind is 100% on board with the idea. Or at least, it was, until:

GiveWell estimates that $3340 worth of donations to malaria prevention saves, on average, one life.
Let us be excruciatingly cautious and include a two-order-of-magnitude margin of error. At $334,000, we are super duper sure we are saving at least one life.
So. Say I’m a millionaire with a spare $334,000, and there’s a guy I really don’t like…

 (Scott further specifies that you are a master criminal that will never get caught, it looks like death by natural causes so you don't waste police time, etc. or that you further offset those costs with more and more donations, as one in principle could)

So. As is its wont, my brain broke down on that one. One part of my mind says "Well, which world would you rather live in? The one where this mysterious millionaire didn't save all those lives, at the expense of killing one person? By any reasonable standard, that's a better world to live in: if death is bad, then saving lives is good, and saving more lives is better" The other part mostly yells "but murder is bad!".

The key insight here, as far as I can tell, is that my intuitions on morality break down somewhere in the vicinity of murder. I can be OK with the idea of killing one person to save many others (e.g. the trolley problem) because you didn't put the person in the tracks. I can even be OK with the fat man version of the trolley problem, because it's not your fault that's the only way to save five people. But I'm not OK with this. Where's the difference?

The obvious candidate is "But you have another available course of action: not murdering anyone, and donating the money anyway. That's clearly better." And that's true, and if true it applies equally to all ethics offsets, not just re: murder. And I agree: if it comes to me, the obvious ethical decision is not to murder anyone and donate almost all my money to the most efficient charity. No question. But people don't actually do the most ethical action if it's too inconvenient. 

Suppose I am building an ethics system to be used by imperfect humans, and some of those humans happen to be murderous millionaires. Suppose that those murderous millionaires would obey "don't murder anyone" as a rule, and would also obey "If you want to murder someone, donate X amount of money to charity to offset you murder", but they would not accept "Don't murder anyone, and also, donate all your money to charity". Sitting in this position, it seems to me, my brain can relax and think right: this is a trolley problem. The trolley was set in motion by some very peculiar quirks of the psychology of hypothetical millionaires, but it's no less trolley-ish. I still have several lives on one metaphorical track, and one on another. Sucks for the one.

I'm not sure what this means for the problem I discussed last post, (i.e. a good way to ground "killing people is bad" without resulting in life maximization), other than further confirmation that I can't trust my intuitions on killing people to be consistent.

Death is bad (but I'm not sure why)

(thoughts prompted by this post on utilitarianism and abortion)

I would urge you to read the linked article on its own, especially if you self-identify as an effective altruist or utilitarian or somewhere on that philosophical area. But for the purposes of this blog, there's an argument there that goes like this: Murder is very bad. Most people who support abortion are sure that killing a 1st trimester foetus is not murder, but they should also be aware that there a lot of people who disagree with them. Therefore, they should not be 100% certain* that abortion is not murder. Therefore, if you admit something like a 1% chance that abortion is in fact murder, and therefore very bad, and if you put numbers on "very bad" (that's where the utilitarianism comes) it's very hard to make the math come out "abortion is good". (unless you are dealing with extreme cases like abortion to save the life of the mother/foetus will not survive/etc.)

Political disclaimer: My support for legal abortion has less to do with "abortion is morally good" and more with "abortion will happen anyway but if legal it's safer" and "we should probably give people the right to decide how their body is used as a matter of principle, even if they will decide to do bad things with them". "If it's bad it should be illegal" is not a principle I endorse in the general case. So no, I'm not trying to make or endorse an argument for banning abortion.

Back to the argument. There a number of obvious responses, like:

"I'm not an utilitarian and I don't think you can do math on life and death", in which case I would love to have a longer argument with you on the subject but this post is not the place, or

"I am, in fact, very much certain that foetuses are not people and killing them is not murder, less than 0.01% chance I'm wrong", or

"Well, obviously in conclusion abortion is wrong", which I think are both interesting positions and I'll address in a moment.

In the "put numbers on very bad" part above, the author uses Quality-Adjusted Life Years (QALY). The argument is that if it turns out abortion is murder, it costs the foetus ~76 years of life, on average. Here's the part where my brain, and my philosophy, goes nuts:

What if a foetus isn't a person? Aren't we missing out on exactly those same ~76 QALY anyway?

There's a philosophy that says no. You can only care about real people, not potential people; those 76 QALY only matter if the person who would have lived them already exists.

If you accept that, yay. You can go back one level to the previous argument and try to figure out the expected personhood of a foetus, which I'm sure must be a barrel of laughs. My problem is that I'm not sure I can.

If I'm certain of anything in meta-ethics, it's consequentialism: the idea that "good" or "bad" is about states of the world. The right action is the one that results in the best state of the world, and nothing else. Not which laws you follow, not which virtues you exercise, just how the world is.

In particular, if "person X exists" is a good state of the world, we should bring it about; if not, we shouldn't. But the "fuck you, potential people" principle says otherwise: If you already exist, then states of the world where you don't mean you were killed, so that's bad. But if you don't already exist, then states of the world where you don't exist are neutral. There's no reason to care about you in the future if you don't exist now.

It seems like a very weird twist on consequentialism: The same state of the world can go from good to bad depending on when you ask the question. That's a very ugly feature I don't really want in my metaethics.

But if you reject that, not only do you have to worry about abortion, but suddenly everything from contraception to not having sex falls in the same bucket: You are not taking action to make a person come into existence, this is the same as taking action to remove a person from existence (since they both result in world states where a person doesn't exist), ergo you are a murderer.

Which brings us to a nasty conundrum: If I want to be remotely consistent about ethics, then either I admit that murder is not always that bad, or I have to stop blogging right now and go impregnate as many women as possible. Since the second option sounds like a lot of work and would probably end badly for everyone involved (except our future children, who are being saved from counterfactual murder!), let's look at the first one.

Why is killing people bad? 

... honestly, I'm not sure. I'm far more certain of the fact of "don't murder" than of any philosophical justifications for it, presumably because hominid brains evolved to have an innate sense of morality where we don't kill each other all the time, because social animals that kill each other all the time don't really work too well.

Like, there's the making people sad argument: if I kill you, your friends and family will be very sad, and making people sad is bad, therefore don't do it. And that's all well and good, except that if that's all it should mean it should be alright to kill people with no friends, or people who have lots of enemies who would be happy to seem the die. It does seem to allow not having sex, since people who don't exist yet don't have friends to care, so that's at least a point in favour.

There's the preferences argument: as a general rule, people's preferences being fulfilled is good, all other things being equal. People prefer not to die, ergo, don't kill them.

But that falls prey to the potential people problem just as well: hypothetical people would most likely also enjoy existing, ergo, if we care about their preferences we should bring them into being. Should we only care about the preferences of people who exist right now? If so, then that raises intriguing questions about the future: are we supposed to stop caring about what happens to the planet after the last currently-living person dies? After all, the people who would be alive then have no moral weight right now. 

It seems to me that I intuitively care about people who don't exist, like, I would think it's very bad if the world a thousand years from now is every human being living a miserable existence. But I don't worry about my potential children not existing. My brain parses "not existing" and "it existed and then stopped" as very different things, even though the end result is the same.

It would then seem there are two choices: 

Existence is not inherently valuable, and I need a good ethical grounding for why murder is bad that I don't have,

or,

We are morally obligated to maximise the number of people who exist, and will exist.

I'm currently defaulting to the first one, hence the title. This just might be because the second one is weird and uncomfortable, and I would really like a good answer for this. But I don't have it.

*As a general rule, you should not be be literally 100% certain of anything, for reasons I may have gone over in the past. Here though, I don't mean just "technically this could all be an illusion created by a trickster demon" but "There is a small but measurable chance you are wrong".

A box of smoking sin

Newcomb's problem is a funny little thought experiment that goes roughly like this: Suppose Omega, the alien superintelligence, comes up to you, scans your brain, and then leaves two boxes on the ground. One, box A, is transparent, and you can see it has $1,000 inside. You can't see what's inside box B. Omega tells you that you can take both boxes, or only box B. If Omega expects you to take both boxes, then it left box B empty, but if it expects you to take only box B, then it put $1,000,000 inside. You know Omega has done this thousands of times and has never made a mistake. Do you take one box, or two?

While I am no expert on decision theory, I know enough to explain how two ways of making the choice, causal decision theory and evidential decision theory, think about this problem.

When faced with Newcomb's, CDT reasons: The boxes are already on the ground in front of me. Whatever I choose now, can't make box B change its contents. Since the contents are fixed, if I two-box I get $1,000 plus whatever's inside box B, if I one-box I get whatever's inside box B. Thus, I am guaranteed to get more money by two-boxing, so I take both boxes. CDT predictably wins $1,000.

In the same situation, EDT reasons: If I look at the money people get by each strategy, one-boxers always get $1,000,000 and two-boxers always get $1,000. If I one-box, I have a probability of ~1 of getting $1,000,000 and ~0 of getting nothing. If I two-box, I have a probability of ~1 of getting $1,000 and ~0 of getting $1,001,000. Clearly, I win much more money on average by one-boxing, so I take only box B. EDT predictably wins $1,000,000.

Now, people first introduced to the problem are pretty split as to what they choose, so I'm not going to say "obviously you can see how X is wrong" about either. But it is worth remarking that a) EDT predictably walks out with more money, and b) there is a some consensus among decision theorists that the "rational" thing to do is to follow CDT and two-box.

Now, for those of my imaginary readers that have concluded that obviously EDT is right and CDT is stupid, consider smoker's lesion.

Smoker's lesion is another thought experiment, which goes like this: Suppose you live in a parallel universe where the correlation between smoking and cancer is not because smoking causes cancer. Rather, it's because there's a gene that makes people more prone to lung cancer and also makes them more likely to enjoy smoking. Suppose you like to smoke, but not getting cancer is far more important to you. In this parallel universe, do you smoke?

CDT reasons: Either I have the gene, or I don't. Smoking is not going to give me the gene, it'll just be doing something I enjoy and will have zero effect on my chances of getting cancer (in this parallel universe). So I should smoke.

EDT reasons: People who smoke are more likely to get cancer than people who don't. So, if I don't smoke, I'll be less likely to get cancer, and thus I shouldn't smoke.

I don't know how people's opinions split on this one, but I'd guess that most people realise EDT is wrong here. Not smoking doesn't change your genes! You're just missing out on something you enjoy (by the scenario's specification) in the delusion it'll lessen your chances of cancer, which it won't, since there's no causal relationship.

So yes, I smoke in smoker's lesion. Does that mean I should worry that I walk out of Newcomb's Problem with less money? I suppose one answer could be that I don't expect to find an alien superintelligence that can predict my decisions, but smoker's lesion-type situations are more likely. Or that Omega unfairly rewards irrationality and there's nothing I can do about that. (I've heard that last one plenty of times). But those answers are mistaken, for a variety of reasons. My answer is much simpler; you would be able to predict it if you had read the paper I link here (and if "you" actually existed instead of being one of my imaginary readers). I one-box.

Which is to say, I think CDT is wrong in Newcomb's, though I believe EDT is even wronger in smoker's lesion. It's not that I alternate based on convenience, rather, I think something else called timeless decision theory is correct. Again, see here.

Of course, this is hardly my own original thinking. Pretty much everything I said above is taken as obvious in some circles, at least to the extent I didn't make any mistakes. This post is here because of a third thought experiment, this one about sinning Calvinists, which I found here:

John Calvin preached the doctrine of predestination: that God irreversibly decreed each man's eternal fate at the moment of Creation. Calvinists separate mankind into two groups: the elect, whom God predestined for Heaven, and the reprobate, whom God predestined for eternal punishment in Hell.

If you had the bad luck to be born a sinner, there is nothing you can do about it. You are too corrupted by original sin to even have the slightest urge to seek out the true faith. Conversely, if you were born one of the elect, you've got it pretty good; no matter what your actions on Earth, it is impossible for God to revoke your birthright to eternal bliss.

However, it is believed that the elect always live pious, virtuous lives full of faith and hard work. Also, the reprobate always commit heinous sins like greed and sloth and commenting on anti-theist blogs. This isn't what causes God to damn them. It's just what happens to them after they've been damned: their soul has no connection with God and so it tends in the opposite direction.

Consider two Calvinists, Aaron and Zachary, both interested only in maximizing his own happiness. Aaron thinks to himself "Whether or not I go to Heaven has already been decided, regardless of my actions on Earth. Therefore, I might as well try to have as much fun as possible, knowing it won't effect the afterlife either way." He spends his days in sex, debauchery, and anti-theist blog comments.

Zachary sees Aaron and thinks "That sinful man is thus proven one of the reprobate, and damned to Hell. I will avoid his fate by living a pious life." Zachary becomes a great minister, famous for his virtue, and when he dies his entire congregation concludes he must have been one of the elect.

If you were a Calvinist, which path would you take?

This problem I found a bit trickier than the previous two (though, to be fair, Newcomb's was much more confusing before I learned about TDT). So, in the interest of testing my understanding of the principles behind each decision, I ask myself: is this analogous to Newcomb's, or to smoker's lesion (or neither)? What makes them different? Why is it that I am certain that imitating the choice of those who end up better off in Newcomb's is a good idea, but I find it laughable in smoker's lesion?

Think about this: In Newcomb's, you want to be the kind of person who chooses to one-box, because one-boxers have a box with one million dollars there for the taking. (In fact, a CDT who knows in advance they will face Newcomb's problem, would choose to pre-commit to one-box if the option was available). In smoker's, you want to be the kind of person who doesn't smoke, but only to the extent that such a thing means you don't have the cancer-causing gene. Indeed, you could use, say, a nicotine patch to get rid of your desire to smoke, but it would be pointless as a cancer prevention measure and thus have no particular reason to do so.

Could something similar be said about Newcomb's? At first, it might appear that this would be equivalent to someone giving you a brain surgery to make you a one-boxer, but only after Omega left and the box's contents are fixed. But wait. Omega is a superintelligence who can perfectly (or near-perfectly) predict your decisions. If you had the choice to self-modify to one-box, then Omega would know of this, and also know if you would go with it or not, and fill the box appropriately. Perhaps if Omega Prime, the even more powerful superintelligence, had intervened and offered to make you a one-boxer but guaranteed that Omega didn't know of this, that would be analogous. But there's nothing in your own power you could do that would make you a one-boxer without also meaning that one-boxing was a viable strategy, since Omega would predict your self-modification.

This is key. In smoker's lesion, making yourself a non-smoker through any means other than changing your cancer-causing gene is pointless. In particular, choosing not to smoke falls under that category. In Newcomb's, making yourself a one-boxer through any means that are predictable by Omega is not pointless. In particular, choosing to just take box B falls under that category.

So, in Calvin, is choosing not to sin useless or not? Well, here I am unsure about the actual theology, so let it be clear that whatever I say applies to actual Calvinism only to the extent that it is correctly represented by the description of the problem. That aside:

The problem specifies that a sinning or virtuous nature are not the cause of damnation or blessing. God does not use his omniscience to see if I will be a sinner or saint and determine my eternal fate accordingly, rather he determines it based on something else entirely, or random chance, or whatever, and then as a result I act however it is I do. Thus, choosing to be virtuous will not mean that God predicted I would be virtuous, and dictated my fate accordingly. It just means I'm acting virtuous. If I was damned, I'm still damned, just wasting my time not sinning.

One caveat, though. The problem is specified in terms of always-never, the damned always sin, the virtuous always lead righteous lives. Which means that any universe in which I am simultaneously damned and not sinning is in contradiction. This is because the fact of whether I sin or not is, in the problem, specified not by my decision theory but by my connection to God. This, I contend, means it's not a problem where decision theory matters. If the problem simply said that there is a high probability of sinning (as high as you want except exactly 1), then you could use decision theory. This is not an attempt to weasel out;. my decision in both cases is to sin. It just happens that in the first case, my decision cannot affect my behaviour in that particular aspect, by the specifications of the problem.

And before you say the same is true of Newcomb's: The specification is that, out of thousands of tries, Omega has not yet made a mistake. This does not need to correspond to Omega being incapable of failing, only to its chance of failure being small (presumably less than one in several thousands). The same principles hold if Omega is right 90% of the time, or 60% of the time for that matter. (the average one-boxing payout would be 60% of $1,000,000, which is more than the average two-boxing payout, 60% of $1,000 + 40% of $1,001,000). Phrasing the problem in those terms doesn't change the decision-theoretic answer, but it requires a type of mathematical thinking some people tend to ignore.

Further thoughts on morality

Earlier, I discussed my view of morality as a massive multiplayer prisoner's dilemma. Recently, I read quite a fascinating paper on something called Timeless Decision Theory. I knew a bit about it from Less Wrong, but extensive reading about it, in a more complete format, both helped me understand what it's about and prompted more reflection on the subject.

Warning, TDT's suggestions in, amongst other things, some instances of the prisoner's dilemma, go against the predominant views in decision theory, which follow something called causal decision theory. While I find the arguments for TDT extremely persuasive, do keep in mind this is not (yet) settled science.

The relevant version is something called the Twin Prisoner's Dilemma. In it, the person you're playing against is not another prisoner, but rather an exact copy of yourself. As usual, you both have the option of cooperating or defecting, and you must make your decisions without any interaction. Both cooperate, 3 years of sentence each. Both defect, 4 years of sentence each. If one defects and the other cooperates, the defector gets only 1 year and the other one 6. Presumably, you both want to minimise the time you spend in jail.

Now, according to CDT, making the other player an exact copy of yourself changes nothing. No matter what the other player chooses, you get less time in your sentence by defecting, so the "rational" choice, they argue, is defecting. And of course the other guy reasons similarly, both defect and both hit the globally worse situation.

Now, I argued when I first thought about this, if the other player is an exact copy of yourself you can expect, with reasonably high probability, that whatever decision you'll come to, the copy will decide the same. This rules out the "one cooperates-one defects" scenarios, meaning you're left to choose between both cooperate and both defect. And since both cooperate is better, globally and individually, the rational choice is cooperating.

This is exactly what TDT says. Specifically, when dealing with a problem which contains an element that outputs the same as your decision algorithm, you should decide as if your choice determined the output of that element. For example, your exact copy who will decide the same as you do, or a sufficiently advanced brain scanner which can determine what you will decide before you do.

CDT says that, if you making your decision doesn't cause your copy to act as you do, then you can't decide as if was determining. I think CDT is wrong about this, but you really should read the paper linked above for an exhaustive analysis of why TDT works better.

Now, in day-to-day life you aren't going to find exact copies of yourself, so this might seem like a silly thought experiment and nothing else. Some might argue that human thought is so random they are unpredictable. I strongly disagree with that idea. If the way humans make decisions is actually unpredictable, then how the hell do we manage to make correct decisions?  Surely people, being thinking beings and not dice, actually base their ideas on something? If I offer you a million dollars vs a thousand, I can predict that you will very likely take the million. That's not random. Sure, we make mistakes, more so the more complicated the problem becomes, but it is possible to approach a systematic way to make correct decisions. That is, in fact, the whole point of decision theory.

So, suppose you are in the real world, where we cannot have exact copies of ourselves yet. What we can have, though, is people who approach decisions more or less rationally. Suppose two people who both implement timeless decision algorithms, and are each aware have common knowledge of that fact. Put them in a typical prisoner's dilemma scenario. They both know that their decisions are the result of the same algorithm, so they each know that they are in the same situation as in the twin dilemma, and TDT says cooperate in the twin prisoner's dilemma. So they both cooperate, for both individual and global maximum utility.

Now, if that doesn't blow your mind with meaningfulness, pretend you're me and that you see the prisoner's dilemma as one of the big things that get in the way of a good society. Think about it. A society of rational timeless decision theorists cooperates naturally, without the need of an outside enforcing authority, because they know the choice is between all cooperate and all defect. I cannot properly emphasise the meaning of this.

I realise that you can't reduce humans to simple decision algorithms. But the point is not suggesting that we are, it's showing what rationality really means. It's about dispelling the myth that selfish rationalists will take the path of everyone for themselves and collapse civilization.

It's about how, as always, if your smart choice predictably and systematically underperforms another, stupider choice, then it's probably not that smart.


Edit: I noticed I made a mistake in the setup of the problem. It's not enough that both parties know that the other implements a timeless decision algorithm, since the guarantee that they'll arrive at the same conclusion only matters if they both know this fact. Which is to say, they have to not only know that they are both TDT, but also that their knowledge of each other is identical (in the aspects relevant to the decision). I think, anyway, the moment the problem hits enough levels of recursion I get lost.

Naturalism

Yesterday's post was but a long-winded introduction to the idea of prior probability and how even after seeing a piece of evidence that, by itself, favours one hypothesis, you can still prefer another one. I promised I'd get to the point today, and I occasionally keep my promises, so...

The point is thus. A few weeks ago, I came across a paper on ESP, which can learn more about here or read here. Now, I don't believe in ESP. In fact, I find the idea ridiculous. But I also find the idea of dismissing an idea due to its apparent ridiculousness without considering the evidence ridiculous. I do try to be rational, and recently, as you may have read about in older posts, I've realised that I'm not as rational as I thought. And learnt quite a few new things relevant to that, including the concept of Bayesian probability which is rather central to this series of posts.

When first faced with this paper, I'm ashamed to admit I lapsed into old ways of thinking, fell right into the grip of confirmation bias, and started to poke this paper in ways I wouldn't poke one that agreed with me. Not because of any real reason, just because it disagreed with me. That lead me to some rather idiotic conclusions which I, fortunately, decided not to share with the world. 

Later, thinking more clearly and reminded of my commitment to question ideas equally whether they agree with me or not, I basically slapped myself in the face (metaphorically) and said, ok, let's stop for a second and think. Am I really proceeding with this the right way? No.

And thus I now accept the reality of psychic powers.

Nah, of course not. ESP is still bullshit. And the concept of prior probability is quite relevant as to why.

This is actually touched upon in the article I linked, one that I hadn't really taken a good look at before today. Basically, after I sat down, stopped try to rationalise, and cleared my thinking, I remembered what little I know about prior probability, and the concept is quite relevant. Given what we know about how the human brain and the laws of physics work, it is absurdly improbable for information to travel backwards in time in the manner proposed by the paper's author. Special relativity says that it is impossible for information to travel faster than the speed of light. Physics is not complete, of course, so it's not impossible that new quantum effects will show some minor possibility of time-travelling information. But it is very unlikely, and it is incredibly unlikely that the first piece of evidence for that would come from human brains looking at pictures and not, say, particle accelerators. Human brains are hardly ideal environments for quantum experimentation.

The article brings up the example of casinos which is less compelling but more approachable. Basically, if ESP worked as reliably as the paper indicates, it'd be trivial to beat the house, and yet casinos make a profit. There's also the fact that James Randi still has his million dollars. And many other things you could probably think of on your own. Before reading the paper, the prior probability of the ESP hypothesis was so low that basically any hypothesis that fit the data and wasn't ESP would have a huge advantage. Not because of bias, but because of the fact that, based on virtually every observation, reality doesn't seem to work that way.

In fact, based on that knowledge, I made an advance note to myself. I should have taken the effort to write down a proper prediction and test my beliefs, and I did not, but, when considering what the new evidence would led me to believe my conclusion was that ESP was almost certainly not the correct answer, and very probably it would be some methodological error in the experiment. I could not clarify which one, I am not strong enough in the ways of science yet. But, if you trust the rationality of the author of this, they can.

So, I was right, somewhat, which makes me feel a little better over my irrational collapse. I still need to watch out more carefully. But that is not the conclusion I had in mind, since I only read that today and the series was started yesterday. 

Some accuse science of limiting itself to naturalistic explanations, or say that science requires an assumption of naturalism to work. They are wrong. Naturalism is not an assumption, it's a result. Richard Carrier defines naturalism as the idea that every mental phenomenon can be reduced to entirely nonmental ones. I agree with him on this, and it underlies my point. There's nothing preventing science from working if, in fact, nonreducible mental phenomena exist. If there is actually some sort of fact about the human mind that makes it defy physics, that is not beyond the purview of science. Science has pretty convincingly piled the evidence in favour of naturalism, giving it an enormous prior probability. That is why science prefers natural hypotheses to supernatural ones. Not bias, simply Bayesian probability.

Prior probability

Suppose someone flips a coin thirty times, and it comes up heads every time. Suppose, further, that you are asked to choose between several hypotheses regarding the coin. Now, one usually starts by assuming a coin is fair, or close enough, but the odds of thirty heads in a row with a fair coin are roughly one in a thousand million (one in a billion, for you short-scale users). It's not impossible, of course, but one might consider other hypotheses. For example, the "two-headed" hypothesis, which would predict heads in every flip of the coin. With a fixed coin,  you would expect heads at every turn with probability of almost 1 (1 is probability talk for what's most commonly called 100%. You never assign anything probability of exactly one).

While the fixed coin hypothesis is not the first one you consider, the evidence (the observed results of the coin-flips) favours it, so you might decide you venture a guess that you think it's the best hypothesis so far. But, imagine that soon afterwards, your friend comes up to you and says: "Aha! I have conceived a new hypothesis!" (your friend has a love for drama). "The coin is fixed so that it shows heads for thirty flips, and on the flip number thirty-one it shows tails! The evidence of the coin flips supports this hypothesis just as well as your two-headed one!"

You might be tempted to simply ask the coin-flipper to flip the coin again, and yes, that would be a way to test that hypothesis. It is in general a good idea to test that which is testable, after all. But what if you no longer had access to the coin? What if your friend had said a thousand flips, instead of thirty? Would both hypothesis be equally likely, since they are equally supported by the observed coin-flips?

The answer is, rather obviously, no. There's a reason we don't go about suggesting stuff like grue and bleen. Well, those of us that aren't philosophers, with them all bets are off. Anyway, that reason is called Ockahm's Razor, and it says that we should prefer simple explanations over complex ones.

That is, of course, a rather vague definition of Ockham's Razor. You can look up stuff like Kolomogorov complexity and Minimum Message Length for an understanding of what complexity is and how exactly the Razor tells us to avoid it. You could also look up overfitting to see why we prefer simpler explanations to more complex ones that might even be slightly better at fitting the available data.

So, as you would have noticed if you had read all that stuff I told you to last paragraph, "Thirty heads and then one tail" is a more complex hypothesis than "always heads" ("always heads" is in turn more complex than "fair coin", though that is less obvious). Ockham's Razor, amongst other things*, tells us that when you have the same evidence, the less complex hypothesis is more probable. That applies to preferring "always heads" to "thirty-then-one", but it also applies to preferring "fair coin" to "always heads" at the start. That is, we can use Ockham's Razor to tell us which hypothesis is more likely before we gather the evidence. This should give you an idea of what is meant by the concept of "prior probability".

I could define "prior probability" in Bayesian terms, but I'm not good enough at explaining that. I'd just tell you to read this and come back, except that if you already read that I have nothing else to say here. So, I will attempt a brief description of the idea and hope I don't fuck it up too much. Essentially, prior probability is how likely you consider something to be, before weighing a relevant piece of evidence (for or against). After weighing the evidence, your new probability estimate is called "posterior probability". Your posterior probability after one piece of evidence become your prior probability for the next piece. Which means that prior probability is not always calculated without any evidence, it only reflects how your beliefs look at one point in the process of examining new evidence.

So, to combine both concepts discussed so far, Ockham's Razor affects your prior probability. In fact, if you're perfectly rational (you're not) and find yourself in a state of no evidence at all regarding something, Ockham's razor determines your prior probability. Not that you are likely to find yourself in a state of literally no evidence, which is why I emphasise that "prior" in prior probability refers to before the evidence to be considered, and not before any and all evidence. 

Before you knew the result of the coin flips, you had reason to favour fair coin over fixed coin, and not just due to Ockahm's Razor, because you weren't without evidence. You know about coins, for example. You know that, most of the time, coins aren't fixed, that most coins are close enough to fair. If someone had asked you to predict the number of heads and tails, you'd have gone for 15-15, not 30-0, and that would have been the best bet you could've made given the evidence. Similarly, you also had very strong evidence against the thirty-then-one hypothesis, and not just the fact that there are more two headed coins than thirty-then-one coinds. A coin that can somehow manipulate itself to fit into such a specific pattern is very unlikely given the laws of physics. If both hypotheses were equally complex, you'd still have strong evidence to prefer two-headed, and this evidence went into your prior probability. The further evidence, the coin flips, don't favour one over the other, so your posterior probability doesn't favour one over the other any more than your prior did. Indeed, for comparing two-headed vs thirty-then-one, the coin-flips evidence changes nothing. But it doesn't have to, because your prior probability very much favoured two-headed over thirty-then-one already.

 

This was going somewhere, I swear. But this post is already too long, so the story is to be continued tomorrow. I'm sure all zero of you can't wait.

*It is a common misconception that the Razor only applies with equal evidence.It is not one I wish to perpetuate, so I'll take this footnote as a chance to clarify that formalizations of the Razor also show that a hypothesis that is more accurate (fits the evidence better) can lose out against less accurate but simpler hypotheses. Again, look up overfitting to understand why.

Methods of Rationality

I've mentioned this one in passing before, but it deserves more comment. "This one" being Harry Potter and the Methods of Rationality, a Harry Potter fanfic. Yes, I know. Bear with me for a minute.

While I've been a Harry Potter fan for, erm, quite some time now (roughly half my lifetime) I was never into fanfiction. I think I've read two entire HP fanfics, one of them specifically because it was horribly bad (you probably know this one). So, saying that HP&MoR was the best fanfic I ever read would be very faint praise. Instead, I'll say that it's easily one of the best works of fantasy literature I've ever come across, and that I do not say lightly. Though calling MoR fantasy sounds like an odd fit.

The idea is as follows: Petunia married a biochemistry professor, instead of an abusive plot device, and Harry grew up as a child prodigy with a particular interest in science and rationality. Who then finds out that magic is real and the way he thought the universe worked is not. Bit of a nasty shock, but he got better..

Harry goes to Hogwarts and starts looking at magic with a sceptical eye, bringing some disappointments but also pretty significant discoveries. I'm not going into which ones, but apparent rules of magic are broken and, at age eleven, Harry seems to have potential to be the most powerful wizard/scientist ever. But, you can't just power-up the protagonist and leave the antagonists as they were. That way lies the Pitfall of Sue.

Which leads me to one of my favourite things about MoR, Professor Quirrell. The original Quirrell was a weak pawn of the Dark Side, whose main thing was pretending to be a stuttering nobody while secretly being Voldemort's host (if that was a spoiler, I don't know why you're reading about HP fanfic) . MoR Quirrel is badass. And also, the only wizard that seems to get, on the same level as Harry, the power of Muggle science and rationality.

Many are the literary merits of MoR, and I'm not the best person for enumerating them. I'll just say, f you ever read PS/SS (that being the first Harry Potter book), you'll find MoR hilarious, exciting, and possibly fascinating. But that's only half the reason I'm blogging. MoR is rationalist fiction. The fiction part is excellent, but the rationalist part is really what I loved.

I considered myself relatively good at being rational. I'm an atheist, I don't fall for new age bullshit, I could probably refute most arguments for paranormal phenomena from memory, etc. But, it's easy to be "rational" when the rational conclusion is handed to you on a silver platter. I really didn't know nearly as much as I thought about how I was tricking myself.

I went into MoR expecting mostly familiar arguments, science I already knew about, biases and fallacies I could easily name and give examples of. I was wrong. I found so much more. Just to give the biggest example, I barely knew anything about Bayes' rule and how to apply it before reading MoR. I only had the vague notion that there were a few common probability problems I didn't know how to solve yet. That turned out into quite a significant discovery

Eliezer Yudkowsky, the author of MoR, introduced the concepts covered in the fanfic in LessWrong, a  collaborative blog dedicated to rationality. I've been spending quite a lot of the last few weeks there, and it's been a learning experience of the kind I haven't had in, um, ever. Only my discovery of FSTDT comes close, and that was much more spread out in time, and not quite as powerful. Though it did set up my interest in rationality in the first place, so it can't be totally separated or discounted.

But I digress. If you have even the slightest interest in cognitive bias, science, reasoning, and other related topics, I strongly recommend reading MoR. Odds are, you'll learn something new. Or several somethings.