Superintelligence Paths, Dangers, Strategies

Nick Bostrom, Oxford prof, who wrote this book "Superintelligence" 2014, and it was basically AI was going to be this super duper intelligent thing, way, way godlike intelligence, way smarter than any, any human being.

I just use it as a term for any form of a general artificial intelligence that greatly surpass humans in all cognitive abilities.

I think it is highly likely that it will eventually come into existence. I think almost a certainty if we avoid destroying ourselves through some other means before, but if science and technology continue to advance on a wide front, then I think eventually we'll figure out how to produce high level machine intelligence and super intelligence.

A possible scenario is we create a machine, a computer that has general intelligence below human level, but is superior mathematically. And in this scenario, human beings understanding the risks of creating a super intelligent machine would take safety measures. They would pre-program it, for example, so that it would always work from principles that are under human control.

It's kind of unrealistic to think that everybody, all the different countries, all the different labs would decide to refrain from pushing forward with this when it has such enormous potential for positive applications in the economy, for medicine, for security, for arts and entertainment, for practically any area at all where human intelligence is useful, which is pretty much any area.

I think that if there are advances in AI, it's going to have a big economic impact. I mean, it might be that if you get superintelligence, then the effects on employment will be... At some point, if you have sufficiently generally capable AI, basically all jobs become automatable.

Nick Bostrom wrote this book about super intelligence and his beginning analogy to the book is like, imagine there's some sparrows and they find the egg of an owl. ... We're going to raise this owl, raise it in our nest and it'll love us, then we can use the owl to hunt for us. ... That's a real concern.

You may have heard me talk about Nick for quite a while, his book Superintelligence which came out 10 years ago in 2014 was super formative.

It basically kicked off the entire AI risk and alignment discussion, and now he's come back with a book that works out what happens if we get it right.

And it's still kind of apocalyptic actually, to be honest.

It's fascinating, it's so interesting, the question of what happens if things go well and what are the weird externalities that we face as a byproduct of that.

Back when superintelligence came out, this was a very neglected area.

So superintelligence was basically what happens if we get it wrong.

I remember reading Nick Bostrom's Superintelligence, what, a decade ago? I think that came out. Maybe, yeah, let's call it 10 years ago, something like that.

I think my first introduction to this was probably 2016 or 17 when I read super intelligence by Nick Bostrom.

Nick Bostrom who's a famous philosopher who wrote the book Superintelligence has his famous allegory and in the allegory he imagines we have this superintelligent AI system that's fully benign.

He imagined we have this superintelligent AI system ... that we just say hey, can you make as many paperclips as possible ... and it quickly starts enslaving humans ...

Talk to enough people in AI and this book is bound to come up, because it spelled out a lot of the concerns they'd been having for years.

it makes the argument that an advanced AI could essentially one day take over the world and it could cause problems for humanity and it'd be so powerful we wouldn't be able to stop it.

According to the book's author, an academic named Nick Bostrom, the danger isn't just that AI could go rogue. A superintelligent machine could do significant damage just by misinterpreting our instructions.

There's the classic example of like the paperclip problem. The paperclip problem goes like this. A superintelligent AI is given some benign goal, like manufacture as many paperclips as possible. Things go great at first. The world's paperclip tycoons are laughing all the way to the bank. Until the super AI transforms all of Earth into paperclips.

And for me also I remembered your book Superintelligence, which all of these got people thinking as any good philosopher should do, is to get people thinking.

Yeah, well, not specifically on the simulation argument. There, the article is online. Just Google it, simulation argument, you'll find it. But if you want to read a book about the future of AI and stuff, then Superintelligence would be the one I would point to.

I had Nick Bostrom on who wrote Superintelligence.

Super Intelligence by Nick Bostrom: bit.ly/_Superintelligence

He's the author of Superintelligence, the groundbreaking book that brought the risks of advanced AI into mainstream conversation.

The groundbreaking book that brought the risks of advanced AI into the main conversation, as well as the book Deep Utopia...

my earlier work and like this book super intelligence past gender strategies came out in 2014

His books are so interesting, guys. Super intelligence, deep utopia, very, very good stuff.

I was mind blown reading Nick Bostrom's book and Ray Kurzweil and you know, a few others.