Wednesday, November 10, 2010

Good and evil agents

Did you know that the word Sigmalephian has no Google hits as of this writing? You might wonder why I would bring that up (or just assume I'm crazy and/or and idiot, hypotheses I cannot discard). Well, one of my usual Internet aliases is Sigmaleph, which I'm quite fond of. And "Sigmalephian" seems to be a good word to describe something relating to my person, much better than say "Currentian" or "Mirrassian". Or, gods forbid, my true name, which I must keep hidden lest I grant you mystical powers over my person. In an act of convenient labelling and tautology, I have decided to declare I belong to the Sigmalephian school of philosophy. That is, that whichever my thoughts on any subject, it just so happens that they match the thoughts of this Sigmaleph character, which, as luck would have it, is myself. Does that make sense? It shouldn't.

All of the above is just actually irrelevant to the matters originally prompting me to write this post, I just felt I needed to get that out there (here) at some time and this felt like a good opportunity. The following is indeed Sigmalephian philosophy, but then that's true quite a lot on this blog, and remarking upon that fact has never been necessary or useful for the reading of my mental excretions.

You're still here? Huh. 20 SigPoints for persistence. Since SigPoints cannot be exchanged for anything as of now and for the foreseeable future, your true reward is my rambling. Aren't you excited? Well, so it goes.

One thought that has repeatedly happened upon me is that the basic benefit of good is cooperation and the basic benefit of evil is resourcefulness. Which is to say. On the purely pragmatic aspects and ignoring for now self-image and warm fuzzy feelings, "good" agents have as an advantage the fact we live in a world with other good agents and they are more willing to cooperate with others like themselves. The basic weakness of the murderer is that zie doesn't go against the detective, zie goes against the detective backed by the police department supported by a large part of society. And, the advantage "evil" agents have is that they are have more methods available to them. If there are two different ways to solve a problem and one involves kicking puppies, the evil agent will be able to choose based on their relative usefulness, whereas the good agent has the disadvantage of having to also factor in the ethics of puppy-kicking. This doesn't cut both ways, since the evil agent has no particular reason to prefer evil methods to non-evil ones that work better. A decision algorithm that only maximises strategic merits will on average outperform the one that has to balance strategy and ethics.

Where am I going with this? Well, you might notice that the "evil" advantage is intrinsic to evil agents, whereas the good advantage is beneficial only when there's a perception of goodness. That is, any agent who cares less about ethics than the adversary has the advantage of more options, but good agents that don't reap the benefits of the goodness advantage can exist. What you need is other good agents to think you're good and help you, which can happen independently of goodness. Which brings us to the problem. An evil agent can reap both benefits if it is evil but perceived as good. The reverse does not happen, indeed it kinda sucks to be good and perceived as evil, because you get none of the benefits.

As a brief parenthesis. Yes, this is a simplified model, and I'm not addressing what "good" and "evil" are, which is a pretty deep problem. For the purposes of this model, "good" and "evil" are what the society in context thinks they are. This is not synonymous with actual good and evil (as I understand them), but it's usually close enough in most cases. The whole "murder is usually considered bad among humans" thing. Other simplifications are that it ignores the self-image, conscience and intimidation factors, and possibly others, which are not minor, but don't tip the scale far enough, usually. Bottom line, I think the model works for most cases. I welcome any improvements that keep it simple. But first, read the rest, because there's one major flaw I correct later on.

Onwards. So, imagine an evil agent who thinks zirself very smart. So smart, zie considers zirself able to trick most good agents into cooperation, while still using evil tactics. And thus, the incentive for goodness is gone. Problematic if you want people to not be evil, which you do being a good agent (and if you weren't, you wouldn't tell me, now would you?). Note that even if the evil agent considers zirself to be good, zie can  believe most people are mistaken, and thus still want to trick people, because the advantage is in being perceived to match society's idea of good. It's close enough to true that nobody sees themselves as evil, but people can certainly see themselves not matching the general idea of good, or think that everyone is making such a fuss about that minor thing of killing [insert group here] who aren't really people. Or whatever. Addendum noted (no, this is not the major flaw I hinted at), moving on.


Well, at this point I started to consider solutions to the problem. One noticeable thing is that it shows the appeal of an impossible to trick good agent handing out significant punishments and rewards . Impossible to trick so there cannot be a false perception of good, good to make sure it only cooperates with good agents, and the rewards and punishments have to be huge to outweigh any possible advantage of evil. The idea of the omniscient, omnipotent, benevolent god, in other words. Not a stunning discovery, of course, but it put the ball in a more familiar court. Since I'm fairly used to considering why gods are not good answers to questions, that part of my brain engaged quickly, and I noticed my big oversight.

A general principle to consider: In most cases, if believing X is beneficial and X is false, there should exist a true belief Y that delivers the same benefits. Y also should explain why X is beneficial, but that's tangential to the point. In the universe we live in, the power of knowledge is in the ability to make better decisions. When you're deciding based on incomplete knowledge (i.e. the situation every human being is whenever making a choice), the decision based on knowledge closer to the truth should, on average, outperform the others. There are beliefs that have effects not related to knowledge, like say placebo effect and such, but they are not the predominant case. Which adds up to, you should want to be right. When you find yourself in a situation when you want people to be consistently wrong to make better decisions, there's probably something wrong with your "right".

What I was wrong about, rather obvious in retrospect, is that good agents cooperating better is not purely a matter of being more willing to do so given the perception of goodness. Good agents cooperate better, in part, because of the characteristics of "goodness". That's how goodness came to be in the first place, if there was no advantage to it then it wouldn't have been selected for, the primitive good agents would've lost the evolutionary game to those without goodness. And, separate but more important, it's the deeper why behind  good agents wanting good agents. The more good agents a society has, the better it will do, outweighing the advantages of increased resourcefulness due to evilness. Otherwise, it'd be irrational to want a good society, and I'm trying to show the opposite.

In the end, it all adds up to that, while it might seem that a pragmatist might simply try to fake goodness and reap the benefits of both evil and good, in the long term that's a poor group strategy. People should want to cooperate, not because of fear of a false punishment, not just because that's who they are (though that plays a significant part in what good is) but because it works at group-level.

Which brings us to the second part of the problem, the individual perspective. You might notice that while it's better for the society for all its agents to be good, for each individual agent it still seems preferable to be evil and perceived as good, getting the benefits without the drawbacks. Of course, this individual perspective results in society collapsing. It's the Prisoner's Dilemma, all watch out only for themselves so it adds up to the worst global situation. Which, once again, rings that little bell in my head that says that if your "smart" strategy has consistently worse results than the "stupid" strategy, then it can't be that terribly smart.

One answer is that a truly smart society should be hard to trick. Not omniscience, that's beyond human means, but it seems a necessary application of intelligence is detecting concealed evil and thus acting as deterrent. That's, like I said, one answer, but I don't think the best. Creating agents that want to be good is more efficient if it works, but also more difficult. I'd be wary of genetically modifying humans, for example, while theoretically it could be very useful there's many ways it could go wrong.  But, while it still seems that better answers should exist, the thing I'm happy about is that at least I managed to get to a answer that shows a smarter society works better, not worse.

No comments:

Post a Comment