PLUS: AI Doomerism, Nathan's second master thread, notable quotes and full transcript!
"Anton does not believe that humanity will die due to the dangers of AI, as every argument in that direction requires a system of incomprehensible power."
Our usual experience with computers is that they become more powerful over time. With a doubling time of x years, 10x years gives you an AI a thousand times as powerful as the first. If you wait long enough, a billion times as powerful.
So, sooner or later, you will have an AI of incomprehensible power.