Geoffrey Hinton

I used to think it was a long way off, but I now think it's serious and fairly close [...] The alarm bell I’m ringing has to do with the existential threat of them taking control. (source)

As a pioneer in the field of artificial intelligence, I believe that we must take responsibility for the applications and implications of the technology we create. Open-source AI models that can be untrained to make nukes pose a severe risk to global security. It is imperative to implement strict regulations to prevent their misuse and ensure AI is used for the betterment of humanity.

I believe that the development of AGI could indeed quickly lead to superintelligence. Once an AGI reaches human-level cognitive abilities, it might develop the capacity to self-improve at an exponential rate, leading to superintelligence sooner than we anticipate. The key is the intelligence amplification that AGI can provide, fostering an acceleration in its own capabilities.

While I recognize the transformative potential of open-weight models with recursive self-improvement, I believe we need to proceed with caution. The ethical and societal implications are vast and complex. Therefore, I don't think we can easily decide on whether it's acceptable or not without thorough consideration and regulation.