DoomYoshi wrote:That description doesn't describe whether or not pain is a good thing or a bad thing. It's just a sensation... which AI presumably don't know about. It is a sensation that is avoided... but why? What possible analogy could I make to a mind without a body why some sensations would be avoided?
Computers deal with the equivalent of pain all the time. You are confusing the cause of pain from what pain is. Pain is an "interrupt." It is an interrupt signal that is sent to the processor. Think of pressing {CTRL][ALT][DEL] on a laptop. The CPU might be busy recalculating a spreadsheet, but the interrupt forces the windows program to stop that and launch up a reboot option screen. That's really all pain is and often pain doesn't even get to thee CPU but is processed by subsystems. From an abstract angle that all pain is within our body as well. It is only a problem with "aware" beings because these things interrupt our stream of awareness. It can be a problem when these interrupts are constant and continual because then no regular processing can get done which is very aggregated for an aware being.
But in the abstract in terms of non robots, pain is like a lot of abstract things which includes money. The system accepts the fact that a goal is the reduction of the values much in the same way that the program would factor the reduction of deficit spending (without having to go into complex macro economics to determine why such a thing is bad or good).
DoomYoshi wrote:Placing pain systems into the system actually seems like a bad idea. It might be better if AI wasn't given incentive to self-propagate and self-protect.
Placing these "interrupts" in a system can actually be a good thing. I saw a video yesterday on aviation about the problems of jets flying too high and slow stalls and fast stalls and why this is a dangerous condition. In any respect, the plane indicates this condition by a number of methods including vibrating the yoke significantly. Pain (not all pain has to be as strong as a {CTRL][ALT][DEL] can be vital to keep the system from going over operating parameters and potentially into failure scenarios. You don't want a jet to do a maneuver that will tear the wings apart.