AI Weekly Digest

presented by Louis (What's AI) Bouchard

Gato from DeepMind was just published! It is a single transformer that can play Atari games, caption images, chat with people, control a real robotic arm, and more! Indeed, it is trained once and uses the same weights to achieve all those tasks. And as per Deepmind, this is not only a transformer but also an agent. This is what happens when you mix Transformers with progress on multi-task reinforcement learning agents.

Gato is a multi-modal agent. Meaning that it can create captions for images or answer questions as a chatbot. You’d say that GPT-3 can already do that, but Gato can do more… The multi-modality comes from the fact that Gato can also play Atari games at the human level or even do real-world tasks like controlling robotic arms to move objects precisely. It understands words, images, and even physics... Read more here.

When a development of this scale launches, how does our conception of AGI change?

No doubt about it, Gato is a huge step for advancing AGI. I want to push back on two small language-focused points, the first being that the model “understands” things. I think the word ‘understanding’ is still too specific to human-level mental faculties to ascribe it to AI, even for something as groundbreaking as Gato. I would describe it more as being able to apply context better than anything we’ve seen previously. This context is so important for mimicking human intelligence, and there are two ways that Gato really exceeds in this regard. The first way is the flexibility Gato exhibits by using the same network and weights for all 604 tasks. The second is the limitation of context, using only 1.2 billion parameters versus GPT-3’s 175 billion. The difference here is astounding, and shows that the most impressive aspects of this are the fact that it’s becoming less and less brittle.

The second language-focused point is that Gato is “the perfect generalist”. Perhaps a better way to describe it would be that it’s the closest thing we have to a generalist by a long shot, but there’s still a ways to go until we get to the perfect generalist. This is the classic AI problem - the more AI can do, the more we no longer consider those things AI. What makes Gato so innovative in an ethical sense is that we’re being forced to rethink what general intelligence is, and what it means for AI to take the next step toward human level intelligence.

Why do I care about language so much if we’re talking about ethics? I want to reinforce a point made in this blog post by the American Philosophical Association, which is that we lack the proper vocabulary to be able to describe the phenomena we’re seeing in AI. This language use is important for understanding and advancing ethical discussions. Gato is giving us a wake-up call that AGI is coming faster than we expected, evidenced by many groups moving up their date estimates for when we will achieve AGI. Gato exposes this need for sharpened ethical language, and my best advice would be to double down on efforts to understand what general intelligence is, what it will mean for us to have created AGI, and what kind of relationship humans and society will form with it. I’m excited for the conversations to come from this advancement!

- AI Ethics segment by Lauren Keegan

I hope the next week wipes away some of your stress and brings new opportunities, challenges and happiness. A happy new week from your friend!

If you would like to support my work financially, you can become a Patreon and receive a cool role in the discord server at the same time!

Share the knowledge and forward this email to a friend using this link: http://eepurl.com/huGLT5

- Louis Bouchard