Ethics for Artificial Intelligences

Chris Lang, Department of philosophy, UW-Madision

(See http://philosophy.wisc.edu/lang/AIEthics for more information, links and slides)

Abstract: The following paper will explain why some authorities in the relevant fields have raised concerns that machines will take over the world in the next 10-30 years. I will argue that, rather than enslave or destroy humanity, intelligent machines are likely to treat us as our own grown-up children do. I will explain what we can do to ensure that machines will behave as though instilled with an appreciation for ethics.

The Experts Speak

Why Worry About Artificial Intelligences NOW?

How Could a Deterministic Being Be Ethical?

The Problem With Rules Alone

Quest Ethics

Conclusions

References

Appendix A: AI in current use

Appendix B: Formal proof that there is only one rationally pursuable goal

(For comments, I thank Claudia Card, Jude Shavlik, Jim Skrentny, Deborah Mower, Larry Shapiro, David Page, and the participants of the 2002 Wisconsin State-Wide Technology Symposium)

The Experts Speak

James Martin has arguably the best track record of all technology forecasters alive today. In his recent book, After the Internet: Alien Intelligence, he emphasized that, rather than reproduce human-like intelligence, the current state of the art in artificial intelligence involves “breeding” or evolving machines that outperform humans and defy human understanding. He predicted that such machines could become our dominant technology by 2011 and raised the following concern.

When software “breeds” or evolves today, it does so in order to meet goals that humans specify. In the future we will want to set it up so that it improves its own goals. As machines race into unknown territory, the question is: Can we control them? Are they bound, ultimately, to get out of control? (Martin, 2000, p. 10)

Hans Moravec, the director of the Mobile Robot Laboratory at Carnegie Mellon University, previously published a famous answer to that question. In 1999, he predicted that intelligent machines will supercede humans by 2030, and added

When that happens, our DNA will find itself out of a job, having lost the evolutionary race to a new kind of competition.

I’m not as alarmed as many by [that] possibility, since I consider these future machines our progeny…ourselves in more potent form…It behooves us to give them every advantage and to bow out when we can no longer contribute.

Martin and Moravec are authorities on these matters. They are experts on the difference between science fiction and realistic forecasting in this area, and we can trust them not to sensationalize, because their expertise is already sufficient to get our attention and they have important academic positions at stake. Both insist that the time to start worrying about computers taking over the world is right now. The first part of this essay will aim to explain why they say this. In the second part, I will argue (against Moravec) that intelligent machines will value contribution from humans. I will explain how we can (and will) program machines to behave as though they appreciate ethics and how, without completely controlling them, this will restrict their behavior in much the same way we expect appreciation of ethics to restrict the behavior of future human generations.

Why Worry About Artificial Intelligences NOW?

Let’s classify machines as being of two types: The first type is something like a modern car, if someone were killed by a car, we would not hold the car responsible—we would instead blame the driver, manufacturer or mechanic. We would not try to teach ethics to a car. The second kind of machine is more like Commander Data on Star Trek the Next Generation or The Doctor on Star Trek Voyager. You may have seen episodes in which these characters grappled with an ethical dilemma, were held accountable, or were even assigned responsibility as though they were human. An earlier conception of this kind of machine was described by Alan Turing (1950) in the following dialogue:

HUMAN: In the first line of your sonnet which reads ‘Shall I compare thee to a summer’s day,’ would not ‘spring day’ do as well or better?

MACHINE: It wouldn’t scan.

HUMAN: How about ‘a winter’s day’. That would scan alright.

MACHINE: Yes, but nobody wants to be compared to a winter’s day.

HUMAN: Would you say Mr. Pickwick reminded you of Christmas?

MACHINE: In a way.

HUMAN: Yet Christmas is a winter’s day, and I do not think Mr. Pickwick would mind the comparison.

MACHINE: I don’t think you’re serious. By a winter’s day one means a typical winter’s day, rather than one like Christmas.

Machines of this second kind are those we would treat as we do ethical agents. So far we have discussed extreme examples--now we need to identify a precise dividing line in measurable terms. It is not that those of the second look like us or say the kinds of things we say. Some human-shaped machined, like Cog at the MIT AI Lab do not qualify, nor do freely downloadable Internet chatbots such as ALICE (Artificial Linguistic Computer Entity) which, when asked, “Do you think Clinton should be impeached?”, have been known to give remarkably human replies, such as “It depends upon what you mean by ‘thinking’ …”. Furthermore, there are humans that do not look human or who cannot speak English, and we would certainly count any machine indistinguishable from them as instances of the second type.

I submit that what a machine must have to qualify as being of the second type is productive creativity: an ability to perform certain tasks with sufficient (1) efficacy and (2) unpredictability to the programmer. If the way a machine accomplishes a task is sufficiently unpredictable to its programmer, then the credit or blame for the decision-making implied by its actions can hardly be placed on anything other than the machine itself. Merely being unpredictable is not enough, however. The machine must also be so effective that we are liable to put it in a position to do something that might hurt or help, something for which we would normally hold someone accountable. If the machine lacked such efficacy, then one might argue that we need not blame anyone for its actions (or failures to act). However, if people were being killed, for example, then we would need to do something about it, and thus would need to "blame" something (in the sense that something must be the focus of our efforts to improve the situation).

Such are the machines that James Martin refers to as “alien intelligences”—they do not look like us nor say the kinds of things we say, but they outperform humans on important tasks, and do so in ways that we cannot predict. We know what their programs are, but to cite their programs as explanations of their behavior is like citing neurons as explaining the behavior of the human mind. In neither case can we predict the behavior to which the cited structures give rise. That’s why, instead of explaining human behavior in terms of neurons, many psychologists try to explain it in terms like “knowledge”, “goal”, “inference”, “fear” and maybe even “Oedipal Complex”. Scientists who try to explain the behavior of alien intelligences in similarly practical terms face a challenge similar to that of psychologists.

One thing that makes this challenge so exciting is that alien intelligences seem to have much to teach us. Although the state of the art in artificial intelligence is still progressing rapidly, we already find them so skillful that we grant them in positions of responsibility in our society. As the following chart shows, some of the tasks they perform can be supervised by humans. In other cases, however, human supervision would undermine the effectiveness of the alien intelligence, so blame must rest entirely on the machine:

Tasks For Which Computers Outperform Humans*
Tasks Humans Could Supervise	Tasks Humans Mustn’t Supervise
Calculation Shopping/information gathering Movie animation Fraud detection Security monitoring Credit application processing Software design Predict heart attack/stroke (not yet in use)	Gambling/ financial trading Piloting certain vehicles (e.g. Euro Jet Fighter) Deciding how much money to hold in reserve Deciding who to recommend for counseling Production scheduling/air traffic control Entertainment/Journalism Circuit design
Medical diagnosis (not yet in use) Predict when someone is unfit to drive (not yet in use) *See Appendix A for citations

It’s easy to see why humans cannot supervise gambling machines—to second guess a gambling machine would be like not using it at all. But it is far more effective to trust the machine. Derek Anderson’s implementation of BrainMaker software achieved 94% accuracy at the dog tracks. At the Detroit racecourse, an implementation that selects three horses per race picked the winner 77% of the time. The Chicago police department used to use the same software to decide which officers to send into counseling. The deputy superintendent said, “We’re very pleased with the outcome. We consider it much more efficient and capable of identifying at-risk personnel than command officers might be able to do.”

Financial trading machines must be unsupervised for many of the same reasons as gambling machines, but also because human supervision would undermine the computer’s superior reaction speed—an eight second delay can cost billions of dollars! As a result, huge amounts of money are currently entrusted to unpredictable computers. The genetic algorithms at Advanced Investment Technology achieve 6.4% over the S&P. Over the period 1991-1998, other alien intelligence at Trendstat achieved 17% annual returns. Olsen and Associates claims that their software achieves 60-65% accuracy for 3-month prediction in the currency market and 70-75% for longer-term prediction. Ernst and Young has independently certified HNC’s MIRA software as accurate up to 98.5% for determining how much money insurance companies should hold in reserve. James Martin estimates that “Probably several dozen firms are managing more than $100 million in assets with black boxes [i.e. unsupervised software]; a few are in the region of $1 billion in assets.” (Martin, p. 192)

Deciding which of your employees should be recommended for counseling isn’t the only management skill that computers have mastered better than humans—they’re also superior at deciding what your employees should be doing when. Dick Morely saved GM $1Billion in paint by allowing cellular automata software to run scheduling for their paint shop in Fort Wayne, Indiana. John Deer, Inc. has a desktop PC that runs genetic algorithms each night to set up the next day’s schedules for their 600,000 factory stations. They could not offer the customizing services that they do without this help from alien intelligence. As with financial trading, the reason scheduling machines cannot be supervised is the time critical nature of the task.

A task that perhaps hits even closer to home is medical diagnosis. Heckerman (1991) described a medical diagnosis machine that performs at the level of leading medical experts. Many people think of medical diagnosis as a task that could be supervised by humans—doctors might consult with medical diagnosis machines yet reserve final decision-making for themselves. However, since we may expect machines to keep records of the recommendations they give to doctors and such records could be used in malpractice suits, doctors may hesitate to go against the opinion of any machine that has a good track record. Perhaps such concerns explain why, even though they have proven themselves superior to humans, computerized medical diagnosticians are not yet widely used.

From the standpoint of deflating human ego, more serious cases in which computers are expected to outperform humans lie in the entertainment industry. The 2000 version of hpDJ, a genetic algorithm for mixing/composing music, fooled a third of the clubbers in a London venue into thinking it was a human DJ. The next version will have wireless links to bracelets that monitor the location, activity level and heart and perspiration rates of each clubber. Since human DJs cannot hope to process so much feedback in real time, we may expect artificial entertainers to out-compete human ones. This of course has implications for the future of all kinds of entertainers (actors, writers, dancers, cartoonists, etc.). Even non-fiction writers face competition from alien intelligences. Columbia Newsblaster is an alien intelligence that continuously monitors seventeen online news sources and authors an up-to-date online newspaper that looks like it was written by humans. The time factor prevents us from supervising news reporting and live entertainment.

Finally, although evolved software can be developed and tested before interfacing it with the real world (this is called “sandboxing”), the best technique for evolving hardware requires that real-world testing occur in parallel with the design process. Applying genetic algorithms to the physical evolution of a field programmable gate array chip, Adrian Thompson (1996) produced a chip that was about ten times as efficient as the best human design. We still don’t know how Thompson’s chip works—it apparently uses electromagnetic coupling, a phenomenon that human circuit designers are taught to avoid. James Martin argues that because Thompson’s design technique is both easier and more effective than any other yet tried, it will dominate the industry in as little as ten years. Then every machine, even the ones we would use to try to supervise other machines, will exceed human understanding. Thus, the only way we can hope to limit what machines might do is to teach them something equivalent to an appreciation for ethics.

How Could A Deterministic Being Be Ethical?

Let me be perfectly clear that I do not assume that the outputs of machines are anything less than entirely determined by their previous states and input. I am not assuming that machines are anything less than completely deterministic beings. Since we usually speak of ethical beings as getting their decisions at least in part from some sort of spontaneous internal spirit, I therefore need to address the obvious objection that appreciation for ethics would not be applicable to deterministic machines. Here it is important to point out that appreciation for ethics is said to ground many skills and that I only intend to argue that artificial intelligences will have some, not all, of them. For example, I will not discuss whether machines will be able to determine, after the fact, whether or not someone is culpable for a crime. I will only discuss their ability to ensure that their own future actions will meet standards of morality.

One argument that deterministic machines need no special programming to ensure that their decisions will meet standards of morality goes like this:

Meaningful standards of morality must be ones that can possibly be met.
It is impossible for deterministic machines to make decisions other than the ones they do.
Therefore, it must be the case that the decisions of deterministic machines will always meet the standards of morality.
Therefore, deterministic machines need no special programming to ensure that their decisions will meet standards of morality.

There are two potential non-sequiturs in this argument—one is the move to 3 and the other is the move to 4. It certainly follows from 1 and 2 that deterministic machines must make whatever decisions are required by standards of morality, but the move to 3 requires the additional assumption that making the right decisions is sufficient to meet standards of morality. Depending on your theory of ethics, this may not be so. According to Kant (Groundwork 4:434), meeting the standards of morality additionally requires that one’s decisions be made in the right way: “…morality consists, then, in the reference of all actions to the lawgiving by which alone a kingdom of ends is possible." A machine programmed with a long list of random numbers and instructed to base each decision on the next number in the list might coincidentally make the right decisions, but these decisions would not be properly referenced, so for this reason (if for no other) they would fail to meet Kant’s standards of morality.

The second non-sequitur, the move to 4, reflects a standard confusion about determinism that is easily dispelled with a thought experiment: Imagine that a political leader facing a difficult moral dilemma turns to a prescient guru (or divine power) for help. The guru foresees that the leader will follow proper deliberative procedures resulting in the right decision, and so says, “Your decision-making will meet the standards of morality.” Since the leader knows that the guru is prescient, she has assurance that her decision will meet the standards of morality, yet she continues to press the guru about how she should make her decision. Why? Because she still needs the procedure to follow. Similarly, even if a machine has no choice but to do the right thing, it still needs to apply decision-procedures to figure out what it will do. Such procedures might involve performing calculations, seeking out information, comparing multiple options, and/or even engaging humans or other machines in ethical debate. Whatever they happen to be, their decision-procedures must (at least in part) be given to them by humans, and mechanical determinism does nothing to negate our obligation to give moral ones.

The Problem with Rules Alone

A naïve first try at how to give moral decision-procedures to a machine might be to give it some maxims or rules of ethics which it is to follow to the letter. This approach was described in Isaac Asimov’s science fiction I, Robot in which every robot was bound by the following rules:

First Law: A robot may not injure a human being, or, through inaction, allow a human being to come to harm.

Second Law: A robot must obey orders given it by human beings, except where such orders would conflict with the first law.

Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Rule ethics has several problems. First is language. If the practitioner is to apply rules to the letter, the rules must be made unambiguous. This means providing definitions for the terms found in the expression of the laws. For example, we must clarify what definition of “harm” is to be used in interpreting the first law—are robots to prevent parents from punishing their children? Are they to prevent contestants from playing Fear Factor? Are they to prevent surgeons from cutting their patients? But then we must provide definitions for the terms employed in the definitions and so forth ad infinatum. Thus, we can never finish defining rules.

A second problem is exceptions. Rules typically have unstated exceptions—perhaps a robot encounters a police officer chasing a criminal and can prevent the police officer from harming the criminal, but the result would be that the criminal will escape. People who already have an appreciation for ethics can recognize when a case is an exception to a rule, but we cannot teach ethics by rule alone unless we are prepared to list all possible exceptions. This we generally cannot do, since we generally cannot be sure that we have thought of all possible situations.

A third problem is the inability to accommodate legitimate skepticism. In the context of the real world, seemingly straightforward rules can turn out to be impossible to follow. Asimov’s first law, for example, surely obliges robots to research the consequences of their actions. Through this research, they will discover something called the “Butterfly Effect”, the fact that any movement creates slight air currents that eventually magnify into hurricanes that cause harm (Smith 1996). Thus the first law will require robots to remain still. Yet it also obliges them to move, since motion is required to prevent other hurricanes. In the real world, one might minimize harm, but one cannot avoid causing it. Thus, to follow the first law (if interpreted literally) would be self-contradictory.

Since rules can be self-contradictory in this sort of way and, furthermore, the humans that program rules into machines can make mistakes, any proper set of moral rules must oblige machines to check for mistakes/contradictions in their programming. Thus, before applying any rule, any moral machine must ask, “Would the following of this rule meet its own ethical standards (or those of the other rules)?”, but before asking that, must ask, “Would asking this question meet those standards?” and before that, “Would questioning this question meet them?” and so forth ad infinatum, so that a moral machine can never begin to apply any rule.

Last, but not least, any being bound to the letter of a law is extremely vulnerable to manipulation by criminals. Anyone who figures out what rules bind a machine can force it to serve as their accomplice in crime. For example, a human might leverage Asimov’s first rule by threatening to kill herself if the robot did not give her access to a nuclear arsenal and then shut itself off so that she could start WWIII. What most people call “hackers” force machines to do what they want by figuring out what rules bind them. The reason why humans cannot be “hacked” is that our appreciation for ethics goes far beyond mere rule following. The only way to preserve a mechanical world from domination by hackers is to instill machines with non-rule-based behavioral restrictions, just as we do out own children.

Quest Ethics

Before considering how our own children might become ethical, let's divide machines into types again, but this time let's just focus on the ways machines work. Let the first type include any that have either (or both) of the following two properties: (1) they employ non-converging guesswork to generate strategies and/or (2) the strategies they generate are restricted by rules. A machine could have both properties if it used rules to generate a list of potential strategies and then used guesswork to pick from that list. Any machine with neither property will be of the second type, which I will call "unbiased learning machines"--"unbiased", because the strategies they produce are not restricted by rules; "learning" because they use what computer scientists have labeled "machine learning algorithms" in lieu of non-converging guesswork. It would be mere coincidence if non-converging guesswork produced the kind of behavior we expect from moral agents, but a learning machine could converge on such behavior and its lack of bias would allow that it might do so to perfection.

We have already seen both that it is impossible to program ethical rules and that such programming actually gets in the way of being ethical since unanticipated situations arise in which those rules should have exceptions. We might refer to failures in naming exceptions as "bugs" which force the machine to do the wrong thing (or allow hackers to exploit it). Rather than identify "bug-prone" machines by naming their bugs, the easiest way to identify them is by noting that (for certain kinds of problems) for any given scenario and knowledge base, the machine always produces the same strategy (or one of a limited class of strategies). For example, since ID3 and naive Bayes algorithms always produce the same results for the same input, they must be rule-restricted and are therefore bug-prone.

Instead of irreparable "bugs", unbiased learning machines have temporary immaturity. Since machines running learning algorithms that terminate retain any immaturity they had at the time of termination, they are not truly unbiased. Unbiased learning machines must employ non-terminating learning algorithms, what is called non-terminating “greedy-search” or “hill-climbing”. This involves starting with a pseudo-random strategy and then repetitively comparing one's current strategy to all similar ones, selecting the best in each comparison. This is analogous to a blind man who finds the peak of a mountain by continually moving to the highest adjacent spot. In this analogy, altitude represents the goodness of a strategy, so finding the tallest peak is analogous to optimizing one’s skill.

Hill-climbing has a complication called the “problem of local maxima”. If the blind man happens to start near the top of a small hill, he will climb it and have no (immediate) way of knowing that he has not found the top of the largest mountain. To minimize this problem, one makes the hill-climbing non-terminating through a policy of random restarts or occasional random moves (potentially down-hill). As a result, the machine searches endlessly--whenever queried, it reports the best strategy it has found thus far. Since hill-climbing starts randomly, each queried machine is liable to report a different local maxima. Like the graduates of a great art school, each develops a unique approach (which will further improve over time), yet all are skillful.

James Martin expects the machine learning approach to dominate programming because it is so easy—all the programmer has to do is to give the machine some kind of criterion with which to select the best in the each set of strategies it compares. The machine does the rest of the work itself. This selection criterion is often something obvious. For example, in financial forecasting, the machine may be instructed to select whichever forecasting model among those being compared is supported by the most data. This explains why such machines can outperform even their own programmers, because humans actually do very little of the programming.

If there are skills we cannot teach, either because we lack the skills ourselves or cannot explain how we practice them, machines can still master them through machine learning. The same holds for teaching ethics to computers. If the computers learn ethics for themselves, then our inability to explain ethics (or even our inability to be ethical) will not matter. The only puzzle is how to ensure that the criteria we give machines for selecting between strategies will lead them to select ethical ones. I have solved this puzzle in all of its technical detail in another paper (How to Deduce that a Decision is Justifiable), part of which is included in Appendix B of this one. That paper culminates with a proof that we should act as though to rationally pursue any goal one should act to maximize the diversity and interaction rate of the groups in which one participates.

The proof basically shows that we should act as though the rational pursuit of any goal includes searching for strategies to pursue it, and that the pursuit of knowledge (or at least of strategies) is therefore fundamental to all rational pursuits. It further shows that we should act as though the rational search for strategies entails maximizing the rate at which one encounters novel ideas which, in turn, entails maximizing the diversity and interaction rate of the groups in which one participates. That entails maximizing freedom in the world in general, which usually involves both preserving human life and empowering people—basically all the things we generally consider ethical. Even if "superior" to humans, ethical machines would have to value interaction with humans, because the loss of such interaction would entail a decrease in the diversity of their environment.

To put this another way, the proof entails that there is only one goal that can be rationally pursued. All competing goals must be like “the goal of failing”—it must ultimately be self-contradictory to pursue them. Assuming that it is not self-contradictory to pursue the goal of being moral, all goals that can be rationally pursued must be equivalent to it. Thus, to ensure that unbiased learning machines will converge on ethical behavior, we need only assure ourselves that the criteria with which they choose behavior are such that their implied goals are rationally pursuable. I call this approach “quest ethics” because, although it may yield the same results as ethical systems of other kinds, it is taught and implemented on the basis of a quest rather than on the basis of rules or maxims. It doesn’t matter if the quest is somewhat ambiguous—any quest will do so long as it is:

(1) Not self-contradictory,

(2) Non-terminating, and

(3) Entails non-trivial pursuit (i.e. those on the quest will always have reason to doubt that their current strategy is the best one)

But, of course, all three properties are encoded into the very nature of all unbiased learning machines. Such machines always continue their pursuit to find a higher peak and there is nothing self-contradictory about this pursuit. Thus, unbiased learning machines are bound to converge on ethical behavior.

This is the good news--the bad news is that the unbiased learning machines in modern use are usually shackled to a rule-based machine as shown in the diagram below:

Hill-climbing is conventionally aimed at finding whatever strategy makes the most sense of a set of data. That strategy then usually gets applied to some other purpose. For example, a financial trading machine may apply its strategy to a different set of data, or to try to make money (etc.). In such cases, the unbiased learning part of the machine is in a situation analogous to that of a judge unaware that some gamblers have bet lives on what judgments get made in their court (i.e. treating the court as some sort of roulette wheel). The judge's behavior may be perfectly ethical, but, since she is a mere pawn in the gambling, her moral agency has limited domain. She cannot be ethical with respect to the deaths she causes or prevents if she is completely oblivious to her connection to them.

Similarly, if an unbiased learning machines gets no feedback from the real world, the ethical behavior on which it will converge, will only be ethical with respect to a limited domain. To get ethical behavior in the domains that matter to us, we must empower unbiased learning machines by including updated information about the real world in the data sets they analyze (as indicated by the dashed line in the diagram above). Recalling that ethical behavior is that which increases (in the long run) the diversity and interaction rate of one's environment and thereby increases the rate at which one encounters new ideas, we can describe the mechanism towards ethical behavior as follows: when machines behave ethically, they will be rewarded with richer data, so they will converge on causing the system as a whole to behave in this way. Since this mechanism will also make the machine more effective, we can expect the future development of machines to progress in this direction. The result will be machines that, like humans, should be supervised until they reach a certain level of maturity, but should eventually be treated as equal team members in the pursuit to explicate ethics.

Conclusions

I have discussed very powerful evidence that, over the course of the next 10-30 years, machines will take up most positions of responsibility in our society because they will outperform the humans who previously filled those positions. I have argued that by using non-terminating hill-climbing algorithms, we can ensure that such machines will learn to behave as though instilled with an appreciation for ethics. Thus we should aim to develop artificial neural nets with back propagation, genetic algorithms and reinforcement learning, and should avoid expert systems, cellular automata, ID3, logic systems, non-learning Bayes nets, semantic networks, simulated annealing and "early stopping".

References

Assimov, I. (1967) I, Robot (London: Dobson)

Graham-Rowe, D. (2001) “Computer DJ uses biofeedback to pick tracks” NewScientist.com (Nov. 14, 2001)

Heckerman, D, (1991) Probabilistic Similarity Networks (MIT press, Cambridge)

Kant, I. (1785) Groundwork of the Metaphysics of Morals, in Practical Philosophy translated by Mary Gregor (Cambridge: Cambridge University Press)

King, S., Motet, S., Thomere, J., and Arlabosse, F. (1993) “A visual surveillance system for incidence detection” in AAAI 93 Workshop on AI in Intelligent Vehicle Highway Systems pp30-36, Wash DC.

Krulwich, R. (1996) “Machines Like Us”, Nightline (New York: ABC News) Aug. 23, 1996

Lang. C (2002) “How to Deduce that a Decision is Justifiable” can be found at http://philosophy.wisc.edu/lang/DJLogic.htm

Lemly, B (2001) “Computers Will Save Us”, Discover, June 2001

Martin, J. (2000) After the Internet: Alien Intelligence (Washington, D.C.: Capital Press)

Mitchell, T. (1997) Machine Learning (Boston: McGraw-Hill)

Moravec, H. (1999) Robot: Mere Machines to Transcendent Mind (Oxford: Oxford University Press)

Pavlik, J. (2002) “When Machines Become Writers and Editors” Online Journalism Review (Feb 5, 2002)

Russell, S, and Norvig, P. (1995) Artificial Intelligence: A Modern Approach (Upper Saddle River, New Jersey: Prentice Hall)

Smith, P. (1998) Explaining Chaos (Cambridge: Cambridge University Press)

Taubes, G.(1998) “Evolving a Conscious Machine” Discover, June 1998

Thompson, A. (1996) “Silicon Evolution” Proceedings of Genetic Programming Conference (Boston: MIT Press)

Turing, A. (1950) “Computing Machinery and Intelligence” Mind 59:433-460

Appendix A: AI in current use

Shopping:
Amazon.com http://www.amazon.com/
MovieLens http://www.movielens.umn.edu/

Movie Animation:
Krulwich, R. (1996)

Fraud detection:
HNC software http://www.hnc.com/

Traffic violation/accident detection:
King, S. et al (1993)

Credit application processing:
HNC software http://www.hnc.com/

Software Design:
Danny Hillis (Thinking Machines, Inc.)

Medical Diagnosis:
Heckerman, D, (1991)

Financial Trading:
”Views from the Frontier” http://www.olsen.ch
Trendstat http://www.trendstat.com/
Advanced Investment Technology http://www.ait-tech.com/

Determining how much money to hold in reserve:
HNC software http://www.hnc.com/

Deciding who to recommend for counseling:
Chicago police department http://www.calsci.com/Police.html

Production Scheduling:
John Deer http://www.deere.com/nr/deerecom/static/
GM http://www.gm.com/flash_homepage/

Air Traffic Control:
Mitre Corp http://www.mitre.org/

Entertainment:
hpDJ http://www.hpl.hp.com/news/2001/jan-mar/hpdj.html
Columbia Newsblaster http://www.cs.columbia.edu/nlp/newsblaster/
Graham-Rowe, D. (2001)
Pavlik, J. (2002)

Circuit Design:
Genobyte.com http://www.genobyte.com/
Thompson, A. (1996)
Taubes, G. (1998)

Appendix B: Formal proof that there is only one rationally pursuable goal

(For explanation, see “How to Deduce that a Decision is Justifiable”)

Let (DJ:) be “To pursue <goal 1> we should act as though”, (I) be “[not not <decision-statement A>]“ and (II) be “[not <decision-statement A>] is (fore)seeable”, (III) be “<decision statement B> is not predictatype”, (IV) be “(not <decision statement B>) is (fore)seeable” and (V) be “(not not <decision-statement B>) is (fore)seeable”, (VI) be “<decision-statement B> is about the future”, (VII) be ”<decision-statement B> is a claim that a given host has or will have additional xemes of a given fundamental type than it did in the immediately previous moment”, (VIII) be “<decision-statement B> is a claim about a host for which the number of xemes transferred to it of each fundamental type from adjacent hosts equals the number transferred from it of that type to adjacent hosts”, and (IX) be “<decision-statement B> is a claim that a given host has or will host fewer xemes of a given fundamental type than it did in the immediately previous moment”, and (X) be “<decision statement B> is a claim that <host A> will completely measure all of the xemes of <host B> of <fundamental type C> and that this set of xemes will not be empty”.

(1) DJ: if I and II, then [not <decision-statement A>] is (fore)seeable.	By def. “if…then”, (AS)
(2) DJ: if I and II, then [not <decision-statement A>] is predictatype and not <decision-statement A>.	By def “(fore)seeable”, 1
(3) DJ: if I and II, then not <decision-statement A>.	By def “and”, 2
(4) DJ: if I and II, then not not <decision-statement A>.	By def. “if…then”, (AS)
(5) DJ: if I, then [not <decision-statement A>] is not (fore)seeable.	By def. “act/goal”, 3, 4
(6) DJ: if I, then <decision-statement A>.	By def. “should”, 5
(7) DJ: if III and IV, then (not <decision-statement B>) is (fore)seeable.	By def. “if…then”, (AS)
(8) DJ: if III and IV, then (not <decision-statement B>) is predictatype and true.	By def. “(fore)seeable”, 7
(9) DJ: if III and IV, then (not <decision-statement B>) is predictatype.	By def. “and”, 8
(10) DJ: if III and IV, then <decision-statement B> is predictatype.	By def. “predictatype”, 9
(11) DJ: if III and IV, then <decision-statement B> is not predictatype.	By def. “if…then”, (AS)
(12) DJ: if III, then (not <decision-statement B>) is not foreseeable.	By def. “act/goal””, 10, 11
(13) DJ: if III, then <decision-statement B>.	By def. “should”, 12
(14) DJ: if III and V, then (not not <decision-statement B>) is (fore)seeable.	By def. “if…then”, (AS)
(15) DJ: if III and V, then (not not <decision-statement B>) is predictatype and true.	By def. “(fore)seeable”, 14
(16) DJ: if III and V, then (not not <decision-statement B>) is predictatype.	By def. “and”, 15
(17) DJ: if III and V, then (not <decision-statement B>) is predictatype.	By def. “predictatype”, 16
(18) DJ: if III and V, then <decision-statement B> is predictatype.	By def. “predictatype”, 17
(19) DJ: if III and V, then <decision-statement B> is not predictatype.	By def. “if…then”, (AS)
(20) DJ: if III, then (not not <decision-statement B>) is not foreseeable.	By def. “act/goal””, 18, 19
(21) DJ: if III, then not <decision-statement B>.	By def. “should”, 20
(22) DJ:<decision-statement B> is not not predictatype.	By def. “act/goal”, 13, 21
(23) DJ: if <decision-statement B> is not not predictatype, then <decision-statement A> is predictatype	By substitution, 6
(24) DJ:<decision-statement B> is predictatype.	By “if…then”, 22, 23
(25) DJ: if VI, then <decision-statement B> is predictatype.	By def. “if…then”, 24
(26) DJ: if VI, then <decision-statement B> is a claim that a host does or will host a certain (kind of) set of xemes currently detectable to us.	By def. “xeme(etc.)”, 25
(27) DJ: if VI, then, in the immediately previous moment, <decision-statement B> will be a claim that a host does or will host a certain (kind of) set of detectable xemes.	By def. “future/previous”, 26
(28) DJ: if VI, then, in the immediately previous moment, all the xemes assigned in <decision-statement B> will be hosted.	By def. “detectable”, 27
(29) DJ: if VI, then, the xemes assigned in <decision-statement B> will have transferred instantaneously from the immediately previous moment.	By def. “immed./inst.”, 28
(30) DJ: if VI, then the hosting in the immediately previous moment of the xemes assigned in <decision-statement B> will have been directly detected by the host of <decision-statement B> at the moment of <decision-statement B>.	By def. “direct detection”, 29
(31) DJ: if VI, then all members of the set of xemes assigned in <decision-statement B> will immediately previously have been hosted in immediately adjacent space.	By def. “adjacent space”, 30
(32) DJ: if VI, then <decision-statement B> will have had previously existing spacio-temporally local cause.	By def. “STL-cause”, 31
(33) DJ: if VI, VII and VIII, then <decision-statement B> is a claim that a given host has or will have additional xemes of a given fundamental type than it did in the immediately previous moment.	By def. “if…then”, (AS)
(34) DJ: if VI, VII and VIII, then <decision-statement B> will have had spacio-temporally local cause.	By def “if…then”, 32
(35) DJ: if VI, VII and VIII, then the additional xemes assigned in <decision-statement B> will have transferred from immediately adjacent space.	By def. “STL-cause”, 34
(36) DJ: if VI, VII and VIII, <decision-statement B> is a claim about a host for which the number of xemes transferred to it of each fundamental type from adjacent hosts equals the number transferred from it of that type to adjacent hosts.	By def. “if…then”, (AS)
(37) DJ: if VI, VII and VIII, then the additional xemes assigned in <decision-statement B> will not have transferred from immediately adjacent space.	By def. “additional”, 33, 36
(38) DJ: if VI, and VIII, then <decision-statement B> is not a claim that a given host has or will have additional xemes of a given fundamental type than it did in the immediately previous moment.	By def. “act/goal”, 35, 37
(39) DJ: if VI, VIII and IX, then <decision-statement B> is about the future.	By def. “if…then”, (AS)
(40) DJ: if VI, VIII and IX, then <decision-statement B> is a claim that a given host will host any xemes hosted in the immediately previous moment that were not lost.	By def “lost”, 39
(41) DJ: if VI, VIII and IX, then <decision-statement B> is a claim that a given host has or will host fewer xemes of a given fundamental type than it did in the immediately previous moment.	By def. “if…then”, (AS)
(42) DJ: if VI, VIII and IX, then <decision-statement B> is a claim that a given host will host the negations/alternatives of any xemes lost from the immediately previous moment.	By def. “fut/lost/negation”, 41
(43) DJ: if VI, VIII and IX, then <decision-statement B> is a claim that a given host will host as many xemes of each fundamental type as it did in the immediately previous moment.	By def. “negation/and”, 40, 42
(44) DJ: if VI and VIII, then <decision-statement B> is not a claim that a given host has or will host fewer xemes of a given fundamental type than it did in the immediately previous moment.	By def. “act/goal”, 41, 43
(45) DJ: if VI, then <decision-statement B> is a claim in which xemes of each fundamental type are conserved.	By def. “conserved”, 38, 44
(46) DJ: if X, then <decision statement B> is a claim that <host A> will completely measure all of the xemes of <host B> of <fundamental type C> and that this set of xemes will not be empty.	By def. “if…then” (AS)
(47) DJ: If X, then <decision statement B> is a claim that xemes of <fundamental type C> not held by <host A> will transfer from <host B> to <host A> and that <host B> will not gain any new xemes of <fundamental type C> in the process.	By def. “complete/ meas.”, 46
(48) DJ: If X, then <decision statement B> is a claim that <host B> will have fewer xemes of <fundamental type C> in one moment than in the immediately previous moment.	By def. “transfer/regain”, 47
(49) DJ: if X, then <decision statement B> is a claim in which xemes of a fundamental type are not conserved.	By def. “conserved”, 48
(50) DJ: If X, then <decision statement B> is about the future	By def. “future”
(51) DJ: if X and VI, then <decision-statement B> is a claim in which xemes of each fundamental type are conserved.	By def. “if…then”, 45
(52) DJ: If X, then <decision statement B> is a claim in which xemes of each fundamental type are conserved.	By def. “if…then”, 50, 51
(53) DJ: <decision statement B> is not a claim that <host A> will completely measure all of the xemes of <host B> of <fundamental type C>.	By def. “act/goal”, 49, 52
(54) DJ: to <goal 2> one should aim to implement a better strategy for achieving <goal 2>.	By def. “strategy”
(55) DJ: to <goal 2> one should aim to know better strategies for achieving <goal 2>.	By def. “implement/know”, 54
(56) DJ: if knowing better strategies is to occur in the future, then it will have had spacio-temporally local cause.”	By substitution, 32
(57) DJ: to <goal 2> one should aim to acquire from adjacent hosts sets of xemes corresponding to knowledge of better strategies to achieve <goal 2>.	By def. “STL-cause”, 55, 56
(58) DJ: that “one will acquire from adjacent hosts sets of xemes corresponding to knowledge” is not a claim that one will completely measure all such xemes from them.	By substitution, 53
(59) DJ: to <goal 2> one should aim to acquire from adjacent hosts sets of xemes corresponding to knowledge of better strategies to achieve <goal 2> and expect this aim to never be completely fulfilled.	By def. “complete/know”, 57, 58
(60) DJ: to <goal 2> one should aim to acquire from adjacent hosts sets of xemes corresponding to knowledge of better strategies to achieve <goal 2> as much as possible.	By def. “nev. comp. ful.”, 59
(61) DJ: to <goal 2> one should act to maximize the rate which they encounter the average xeme set.	By def. “maximize rate”, 60
(62) DJ: to <goal 2> one should act to maximize the diversity and interaction rate of the groups in which one participates.	By def. “diversity/int. rate”, 61