Discussion about this post

User's avatar
JGPryde's avatar

Really enjoyed reading this take on the possible upsides of AGI as our saving grace or “Deus Ex Mechina”. It does seem a bit of a stretch (or dare I say a dodge) to paper over the risks of an AI system going to extremes to meet its stated goal by assuming that we can just assign that problem to another AI. What if they begin to collaborate or the first AI is smart enough to outwit the second AI? Well then we create a third AI to protect against that situation. And another and another. It’s AI turtles all the way down.

Also, if declining fertility is the original problem (I suspect that runaway population growth spurs all sorts of social problems that declining fertility could resolve but that’s a different topic.) Then isn’t THAT the problem that AI should be focusing on, not replacing the missing bodies?

Also 2, I suspect that there are other limiting factors to rampant progress that at best, will constrain the progress and at worst, cause a decline in quality of life. These include environmental, climate change issues and just humanity’s capacity to absorb this much progress in a limited amount of time. We’re having cultural battles of bathrooms for God’s sake!

What this really boils down to is the super progressive society as imagined is subject to the classic “we don’t know what we don’t know” peril. Not a reason to stop research but definitely a reason to temper expectations.

Lastly, in the sentence:

“There are good reasons to be concerned about rouge “digital gods” that could threaten our civilization.”

I always thought “digital gods” came in more of a crimson red but they are equally roguish. ;^)

-jgp

Expand full comment
Peter Banks's avatar

Great article!

Expand full comment
4 more comments...

No posts