Nick Bostrom

Developing superintelligence is not like playing Russian roulette; it is more like undergoing risky surgery for a condition that will otherwise prove fatal. [...] Models incorporating safety progress, temporal discounting, quality-of-life differentia...

once we have full AGI, super intelligence might be quite close on the heels of that.

there could well be other systems now, or in the relatively near future, that would start to satisfy the criteria

the greatest existential risks over the coming decades or century arise from certain, anticipated technological breakthroughs that we might make in particular, machine super intelligence,

The ethical issues related to the possible future creation of machines with general intellectual capabilities far outstripping those of humans are quite distinct from any ethical problems arising in current automation and information systems. Such su...