Artificial intelligence agents that explore and interact in virtual worlds represent the next major technological revolution. One of the first AIs to follow this path is called Voyager, an agent developed to learn how to play Minecraft, the popular Microsoft video game. While AIs are transforming multiple industries, Voyager currently has a more specific goal: mastering Minecraft by building houses, hunting animals, and exploring the vast game environment.

What if we set GPT-4 free in Minecraft? ⛏️

I’m excited to announce Voyager, the first lifelong learning agent that plays Minecraft purely in-context. Voyager continuously improves itself by writing, refining, committing, and retrieving code from a skill library.

Unlike other AI agents trained in Minecraft, Voyager uses the advanced GPT-4 model from OpenAI and has the ability to write its own code to improve its performance as it adapts to the game. According to its creators, Voyager learns by exploring the world of Minecraft, using basic tools and rules to discover as many things as possible, much like a human player would when starting the game. In fact, the agent outperforms other AI systems by obtaining 3.3 times more unique items and covering twice the distance autonomously.

 

The AI that Programs Itself

Voyager not only explores the world of Minecraft but also adapts and improves its performance through its self-programming ability. This skill allows it to optimize its decisions in real-time, setting it apart from other AI agents. The team behind this breakthrough consists of students from prestigious universities such as Caltech, Austin, and Stanford, with support from NVIDIA. Together, they have developed this agent to lay the groundwork for the next generation of AI.

Voyager is built on three key pillars: a machine learning plan that guides its exploration, a skill library that defines what it can do within the game, and a feedback system that helps it refine its responses to complex situations. These elements mirror the abilities of a human player, who adjusts tasks based on progress, sharpens skills, and explores the game world with curiosity.