Uploads from Ontology Explained: Philosophy and AI

Currently Playing: Is Building Superintelligence a “Cursed Problem”? | If Anyone Builds It, Everyone Dies Ch. 10

Playlist: https://www.youtube.com/playlist?list=PLSjZcAM1g_Us We love to watch people fail. Sometimes that's just schadenfreude (looking at you, America's Funniest Home Videos), but watching others fail can also teach us hard-won lessons without paying the price ourselves. But what if the lesson we learn is that even attempting a task is a mistake? In Chapter 10 of If Anyone Builds It, Everyone Dies, Eliezer Yudkowsky and Nate Soares argue that trying to build artificial superintelligence (ASI) safely is "an insane and stupid gamble that NOBODY SHOULD BE ALLOWED TO TRY!" Their case: hard engineering problems like space probes, nuclear reactors, and computer security only became safe because we could fail and try again. With ASI, failure means extinction. We get one shot, and we don't know enough to get it right. In this video, I lay out the chapter's master argument and walk through its three case studies: lost Mars probes, the Chernobyl disaster and its four "curses," and the endless whack-a-mole of cybersecurity. Then I put the argument to the test. Where I agree: we should learn from history, and when the stakes are high, it's okay (even admirable) to stop rather than risk disaster. Where I push back: - The argument isn't valid as stated. It also needs to show that we can opt out of the ASI problem without paying a great cost. - The analogies are weak. The cybersecurity and space probe lessons apply to almost every hard problem, so they prove too much. Nuclear power is the best of the three, but our track record there might make us more optimistic, not less. - "We only get one shot" is misleading. The problem can be split into smaller problems, and people have already tried and failed to build ASI, and we're still here. - "We don't know enough" is too general. "We don't know how to do X safely" supports targeted regulation and liability for specific functions, not a blanket ban on AI. - And then there's the Malcolm Gladwell Problem: drawing a sweeping moral from one vivid story like Chernobyl and applying it everywhere. Is building AI a cursed problem? It's certainly a hard one. But "nobody should be allowed to try" is a stronger conclusion than this chapter earns. This video is part of my ongoing series on If Anyone Builds It, Everyone Dies. Check out the earlier episodes for my responses to the rest of the book. 📚 References & further reading Hank Green on Chernobyl: https://youtu.be/hIGtTImeYU4 Chernobyl documentary: https://youtu.be/f5Z-1DfEtAM World Nuclear Association, Chernobyl Accident: https://world-nuclear.org/information-library/safety-and-security/safety-of-plants/chernobyl-accident Brain activity (National Academies): https://www.nationalacademies.org/read/1785/chapter/9#141 Paul Erdős: Mathematical Genius, Human (In That Order): https://studylib.net/doc/7913282/paul-erd%C5%91s-mathematical-genius--human--in-that-order- Paul Erdős obituary (Washington Post): https://www.washingtonpost.com/archive/local/1996/09/24/paul-erdos-an-eccentric-titan-of-mathematical-theory-dies/0644a3e5-36b6-4b29-bb73-4d24ad754628/ If Books Could Kill on Gladwell's Outliers: https://open.spotify.com/episode/2hcNuBrCkw7bJbloxfX4Q6 Gladwell's 10,000 Hour Rule: https://www.sjsu.edu/people/john.estill/courses/158-s15/10000%20Hour%20Rule%20Gladwell.pdf #AISafety #Philosophy #Superintelligence