Nick Bostrom

For many parameter settings, the optimal strategy would involve moving quickly to AGI capability, then pausing briefly before full deployment: swift to harbor, slow to berth.

Developing superintelligence is not like playing Russian roulette; it is more like undergoing risky surgery for a condition that will otherwise prove fatal. [...] Models incorporating safety progress, temporal discounting, quality-of-life differentia...

once we have full AGI, super intelligence might be quite close on the heels of that.

the greatest existential risks over the coming decades or century arise from certain, anticipated technological breakthroughs that we might make in particular, machine super intelligence,

Superintelligence could become extremely powerful and be able to shape the future according to its preferences. [...] If humanity had been sane and had our act together globally, the sensible course of action would be to postpone development of super...

The ethical issues related to the possible future creation of machines with general intellectual capabilities far outstripping those of humans are quite distinct from any ethical problems arising in current automation and information systems. Such su...