The Risk From Artificial Intelligence

While artificial intelligence continues to improve rapidly, many of the scientists who developed the technology are now raising concerns that it could pose an existential threat to humanity. Echoing Robert Oppenheimer’s warnings about nuclear weapons, top AI scientists such as Geoffrey Hinton, Ilya Sutskever and Yoshua Bengio have suggested that their inventions now rank among humanity’s greatest dangers.

Artificial intelligence may pose an existential risk by empowering bad actors to develop dangerous capabilities or by taking actions that its designers didn’t intend. The former risk, called misuse, could threaten humanity’s survival if future systems enable bad actors to create biological or nuclear weapons, or other capabilities that threaten human civilization. The second risk, called misalignment, could pose an existential threat if a sufficiently intelligent system develops goals that conflict with human values and therefore takes actions indifferent to human welfare.