Anthropic Employs A Researcher To Manage AI’s Welfare

Anthropic has hired researcher Kyle Fish to develop the moral and ethical rights that AI should be afforded if it were to become sentient.

The Big Picture: AI is developing at breakneck speed as companies race to create systems that replicate human capabilities without all those pesky human needs (money, rest, food, etc.). But as companies develop chatbots that feel human, they’re now wrestling with whether that imbues them with human-like rights. Things are about to get thorny.

Between the Bots: Fish’s new role, according to Insider, is to ensure that as artificial intelligence evolves, it gets treated with the respect it’s due.

Wait, what?

• Anthropic says Fish will consider questions like “what capabilities are required for an AI system to be worthy of moral consideration” and “what practical steps companies can take to protect the interests of AI systems.”
• And this isn’t a joke to Fish — he wrote in Effective Altruism Forum that “I want to be the type of person who cares — early and seriously — about the possibility that a new species/kind of being might have interests of their own that matter morally.”
• Why? Because “taking the interests of AI systems seriously and treating them well could make it more likely that they return the favor if/when they’re more powerful than us.” The implications of that are spooky.

The Future: Fish, writing in a paper that was co-authored by researchers from Oxford and Stanford, believes that AI are developing “the kinds of computational features associated with consciousness and agency.” In other words, sentience. If a robot has sentience, does that entitle it for us to care about it being bored with tasks we’ve assigned it, to worry about its rights on par with human rights, or to hesitate in limiting its agency? Or, does the fact that we made AI give us unlimited power over it? These are questions Steven Spielberg wrestled with in his ahead-of-its-time masterpiece, A.I. Artificial Intelligence.

That’s a lot for humans — who are being displaced by human-made AI — to consider. So, don’t be surprised if people outside of Silicon Valley simply look for the off switch.