General Purpose Intelligence: Arguing the Orthogonality Thesis

ABSTRACT.

In his paper "The Superintelligent Will," Nick Bostrom formalized the Orthogonality thesis: the idea that the final goals and intelligence levels of artificial agents are independent of each other. This paper presents arguments for a (narrower) version of the thesis. It proceeds through three steps. First it shows that superintelligent agents with essentially arbitrary goals can exist in our universe - both as theoretical impractical agents such as AIXI and as physically possible real-world agents. Then it argues that if humans are capable of building human-level artificial intelligences, we can build them with an extremely broad spectrum of goals. Finally it shows that the same result holds for any superintelligent agent we could directly or indirectly build. This result is relevant for arguments about the potential motivations of future agents: knowing an artificial agent is of high intelligence does not allow us to presume that it will be moral, we will need to figure out its goals directly.

Keywords: AI, Artificial Intelligence, efficiency, intelligence, goals, orthogonality

1. The Orthogonality Thesis

Scientists and mathematicians are the stereotypical examples of high intelligence humans. But their morality and ethics have been all over the map. On modem political scales, they can be left- (Oppenheimer) or right-wing (von Neumann) and historically they have slotted into most of the political groupings of their period (Galois, Lavoisier). Ethically, they have ranged from very humanitarian (Darwin, Einstein outside of his private life), through amoral (von Braun) to commercially belligerent (Edison) and vindictive (Newton). Few scientists have been put in a position where they could demonstrate genuinely evil behavior, but there have been a few of those (Teichmüller, Philipp Lenard, Ted Kaczynski, Shirö Ishii).

Of course, many scientists have been absolutely conventional in their views and attitudes given the society of their time. But the above examples hint that their ethics are not strongly impacted by their high intelligence; intelligence and ethics seem 'orthogonal' (varying independently of each other, to some extent). If we turn to the case of (potential) artificial intelligences we can ask whether that relation continues: would high intelligence go along with certain motivations and goals, or are they unrelated?

To avoid the implicit anthropomorphisation in terms such as 'ethics,' we will be looking at agents 'final goals' - the ultimate objectives they are aiming for. Then the Orthogonality thesis, due to Nick Bostrom (Bostrom, 2012), states that:

Intelligence and final goals are orthogonal axes along which possible agents can freely vary. In other words, more or less any level of intelligence could in principle be combined with more or less any final goal.

It is analogous to Hume's thesis about the independence of reason and morality (Hume, 1739), but applied more narrowly, using the normatively thinner concepts 'intelligence' and 'final goals' rather than 'reason' and 'morality'.

But even 'intelligence,' as generally used, has too many connotations. A better term would be efficiency, or instrumental rationality, or the ability to effectively solve problems given limited knowledge and resources (Wang, 2011). Nevertheless, we will be sticking with terminology such as 'intelligent agent,' 'artificial intelligence' or 'superintelligence,' as they are well established, but using them synonymously with 'efficient agent,' artificial efficiency' and 'superefficient algorithm.' The relevant criteria is whether the agent can effectively achieve its goals in general situations, not whether its inner process matches up with a particular definition of what intelligence is.

Thus an artificial intelligence (AI) is an artificial algorithm, deterministic or probabilistic, implemented on some device, that demonstrates an ability to achieve goals in varied and general situations. …