I’ve wondered about the idea of what I call Killer AI, Skynet and terminators and Agent Smiths and those types of things. Isn’t the idea that the machine would see that humans are a threat to it, and then decide to exterminate the race? What I find puzzling is why would a machine care?
Another model I have in my mind for AI is Mr. Data from Star Trek the Next Generation. Continuing with the idea of a machine caring or not about its existence, many times we saw that Data simply did not care. If you told Data that you were going to switch him off, he would just shrug and be switched off. He had no preference.
My favorite color is blue. Why do I prefer blue over other colors? I just do right? I’m built that way, so to speak. I came out of the box like that.
How was I built? What process built me? Well, I was built by my mother and father who combined their genetic material to create me. I assume that somewhere in that material is a code that causes me to prefer blue. I would further assume that this code conferred some advantage to my ancestors, that caused them to survive and pass the blue code on. So in the larger sense, evolution built me, and evolution has set me up to prefer blue. Perhaps my ancestors were people who prospered by the bright blue ocean.
In any case, what I am getting to here is that the reason I think I have preferences is they give me survival advantages. This brings me to another point. A more important point. I believe I have code in me that gives me a preference for self-preservation. Or perhaps simply causes my system to attempt to preserve itself.
One specific mechanism of preservation is fight or flight. When I sense that I am being threatened, my body reacts by energizing me to either eliminate the threat or flee from it.
From a programming perspective, what does my system have to do to be able to self-preserve?
Well, first it has to have some idea of self. Does it? Perhaps idea is the wrong word. No idea is required I think at all. But somehow the system is designed to maintain its integrity.
I can take a hammer and smash my computer to pieces. It will do nothing. However, if I were to try to take a hammer to my dog though I would certainly not, but if I did, he would run away and hide.
How would I model this with a computer program? I know literally nothing about this area, and I am sure it shows. But I find it interesting to think about. So bear with me, and lack of humility in assuming anyone is reading this.
One rule we could make would be that if a square collides with a circle, the circle is destroyed. That gives us a threat.
At this point, however, the circle does not care. It has no “preference” for existence or nonexistence. How could it be imparted with a preference? That is very difficult for me to think about.
More easily I could program it to avoid the squares. That seems a far cry from giving it a systems level integrity bias that I think of as self-preservation. But it might be a start.
By now though you can see where this is going. You probably saw it long before I did. If we programmed computers to care about things, they might not care in the same way we would care, but the effects would be the same. If we programmed a computer to prefer blue, it would prefer blue. In fact, we can program a computer to prefer blue, and it can find blue for us, even if we ourselves can not see the blue.
I see no reason why a programmable construct could not be designed to maintain system integrity. It could also be designed to avoid threats to integrity and also to attempt to eliminate threats to integrity.
I have thought before about a car that would maintain its system integrity. Let’s assume that for a car, to run out of fuel is to die. A computer in the car could be programmed to keep the car fueled and running. With GPS the computer could tell how far it was from a gas station, it could know how many hours of fuel it had left, and distance and so forth, and when it computed that it would run out of gas, it could navigate to a fuel station.
This is a stretch but think of a human driver who ignores the computer and does not stop for fuel. If the computer preferred to have fuel more than it preferred the wishes of the human driver, it could eject the driver or simply lock the controls and take over entirely.
Now we wouldn’t program a computer to do such a thing, would we? Two final thoughts.
- We certainly would. Yes absolutely.
- Who programmed fight or flight into me? You could say evolution did. Can programmable constructs be programmed to evolve? Certainly, they are now. Once you put evolution into a self-contained system, how do you control it? Have we stopped bacteria from evolving resistance to antibiotics? At this time we have not, and bacteria are pretty dumb.
My conclusion: a very sophisticated evolving AI will get out of control.