Imagine teaching a computer to play a complex video game like a human. That's the challenge researchers tackled in "Aligning Agents like Large Language Models." Instead of relying on explicit reward functions, which can be difficult to design, they borrowed techniques from training Large Language Models (LLMs). Just as LLMs learn to write by predicting the next word in a sentence, these agents learn to play by mimicking human gameplay. The initial training uses a massive dataset of recorded human play, giving the agent a broad understanding of the game. However, like an LLM that sometimes generates unhelpful or nonsensical text, this agent might also learn undesirable behaviors. To fix this, the researchers fine-tune the agent on a smaller dataset of high-quality human play for a specific task within the game. This is similar to how LLMs are fine-tuned to follow instructions or perform specific tasks. But there's another layer of refinement: preferences. The agent is set loose in the game to generate various gameplay examples. Then, a 'reward model' is trained to score these examples based on preferences, similar to how LLMs are trained to avoid toxic or harmful outputs. In the research, the preferences were automated; however, in a real-world scenario, a game developer could guide this reward model, essentially saying, 'I prefer this play style over that one.' Finally, the agent uses this reward model to improve its gameplay, ultimately aligning its actions with the developer's vision. This research highlights the convergence of training methods for LLMs and video game agents. It suggests a future where agents can be trained to act not only effectively but also according to specific preferences, unlocking exciting possibilities for game design and beyond.