Sonia Huang Speaking of the course generation mechanism that you just mentioned, I think this is very interesting because it seems to be one of the problems in the world of reasoning and LLM that has not been fully solved yet. How do you make these models aware of what to do next to improve? Can you talk a little bit more about how you build curriculum generation and reasoning? Jim Fan, definitely. I think these state-of-the-art models show a very interesting emergent property in that they are able to think about their actions, and they seem to know what they have mastered and what they don't yet know, and they can suggest tasks accordingly. In Voyager, we set a high-level instruction for the agent, which is to discover as many new items as possible. We give it just a one-sentence goal, and we have no instructions about which items to discover first or which tools to unlock first.
its own through coding, hints, and a library of skills. It's denmark phone numbers amazing how this system works, and I think it's a feature that comes naturally after having a powerful inference engine. Sonia Huang Why do you think so much virtual world research focuses on virtual worlds? I believe that not only do many deep learning researchers enjoy playing video games, although that probably helps a little bit. How do you think solving problems in a virtual world is related to solving problems in the physical world? How do the two affect each other? Jim Fan Yeah, I always thought there were a lot of similar principles between games and robots. For embodied agents,their input is sensory information, such as video streams and some sensory input, and their output is an action. In a game, that might be keyboard and mouse actions, whereas in a robot, it's low-level motion control.
So, from an API perspective, the two are similar. These agents have to explore the world and, to some extent, collect data themselves. But the difference is that robotics is harder because you have to bridge the gap between simulation and reality. In simulation, physics and rendering can never be perfect, so transferring what's learned in simulation to the real world is a challenge and an open research question. So robotics has a "simulation-reality gap" problem that games don't have. You train and test in the same environment. So that's one of the differences between the two. Last year, I proposed a concept called the Foundation Agent, and I believe that eventually we'll have a model that can be applied to both virtual and physical agents. A basic agent will be able to generalize in three aspects: first, the skills it can perform, second, the embodied form it can control, and third, the world it can master, whether it's a virtual or real world.