Reinforcement learning (RL) shows promise in teaching robots complex tasks, but designing the right reward functions is crucial. These functions essentially tell the robot what constitutes “good” behavior. However, crafting them is often a tedious manual process, even for experts. Recent studies have tried using large language models (LLMs) to generate these reward functions from text descriptions of a task. But words alone aren't enough. LLMs often misinterpret subtle aspects of a task or misjudge the relative importance of different goals. Imagine trying to teach someone to cook solely through text messages—it's just not ideal. Researchers at Georgia Tech have developed a new approach called ELEMENTAL, which combines the power of language models with the richness of visual demonstrations. Instead of just telling the robot what to do, users can now *show* it. ELEMENTAL uses vision-language models (VLMs) to analyze both a text description and a visual demo, perhaps a video of a person performing the task. This combined input helps the VLM identify the key elements the user cares about. Think of it like watching a cooking show where the chef explains and demonstrates each step. The real magic of ELEMENTAL lies in its use of *inverse reinforcement learning* (IRL). Traditional RL starts with a reward function and trains a robot to maximize it. IRL flips this—it starts with expert behavior (the demonstration) and works backward to figure out what reward function would have led to that behavior. This avoids the pitfalls of manually designing rewards and allows the robot to learn directly from human expertise. ELEMENTAL takes IRL a step further by using the VLM-identified features to constrain the possible reward functions, making the learning process more efficient. The system also features a “self-reflection” loop where the robot tries the task, analyzes its performance, and then refines its understanding of the reward function. This allows ELEMENTAL to continuously improve, even without further human input. The researchers tested ELEMENTAL on a variety of simulated robotic tasks, from locomotion to manipulation. The results were impressive, with ELEMENTAL significantly outperforming previous methods, especially in adapting to variations in the tasks. This suggests that combining visual demos with language models and IRL could be a powerful way to teach robots complex, nuanced skills. The next step? Getting ELEMENTAL out of the simulator and into the real world, where robots could learn complex tasks directly from human demonstrations, paving the way for robots that truly understand our intentions.