OpenAI’s ousted co-founder starts safety-focused AI firm

The Future. Ilya Sutskever, the AI legend who helped attract the best talent to OpenAI, has finally revealed his next endeavor — a startup focused on creating an AI system more intelligent than humans with safety as its top priority. While the idea of artificial superintelligence is still only a theory, Sutskever’s startup (if well-funded for decades) could end up building the most beneficial or most dangerous technology of the modern era.

Secret safety
After instigating the firing of Sam Altman as CEO of OpenAI, then helping bring him back, then going silent for months, and then quietly leaving the company, Silicon Valley’s question of “Where’s Ilya?” has finally been answered.

• Sutskever has founded Safe Superintelligence Inc. with Daniel Gross (a major AI investor and former head of Apple’s AI division) and Daniel Levy (a lieutenant of Sutskever at OpenAI).
• The company won’t have any flashy products or other forms of revenue; therefore, there’s no reason to really compete with the likes of the constantly-accelerating OpenAI and Google. It’s essentially just a research organization.
• The startup hasn’t disclosed its backers or how much money it’s raised, but it’ll have offices in Palo Alto, California, and Tel Aviv, Israel.

Safe Superintelligence, which has a vision similar to the original iteration of OpenAI, has a major hurdle to overcome: defining what makes an all-powerful AI system safe. Sutskever likens his definition of safety to be closer to “nuclear safety” as opposed to the typical tech meaning behind “trust and safety.”

Additionally, Sutskever says that the model must not harm humanity on a large scale and will need to be trained on “maybe the values that have been so successful in the past few hundred years that underpin liberal democracies, like liberty, democracy, freedom.”

You know, the basics.