Training robots to perform complex tasks traditionally requires painstakingly crafting reward functions that tell the robot what behaviors are desirable. This reward engineering is a major bottleneck in robotics. Imagine, instead, simply showing a robot what you want it to do and having it learn from your preferences. Preference-based reinforcement learning (PbRL) aims to do just that, but getting real-time feedback from humans is often impractical. New research explores how large language models (LLMs) can offer a solution. Researchers have discovered a fascinating way to use LLMs not just to *judge* a robot's performance but also to *imagine* better ways for the robot to achieve a task. This innovative technique, called RL-SaLLM-F (Reinforcement Learning with Self-Augmented LLM Feedback), allows robots to learn complex manipulations, like opening drawers and pressing buttons, *without* any pre-programmed rewards. The LLM acts as a virtual instructor, comparing pairs of robot trajectories and deciding which one is closer to achieving the goal. More intriguingly, the LLM then uses its generative capabilities to create an “imagined” trajectory that's even better than the best real one it observed. These imagined trajectories provide valuable training data, allowing the robot to learn much more efficiently. Experiments show robots trained with this LLM feedback can achieve performance comparable to, and sometimes exceeding, robots trained with traditional, reward-based methods. While the LLM’s judgment isn’t perfect, with an accuracy around 70% compared to ideal reward functions, this research opens exciting new avenues for reward-free robot training. This approach is not without its challenges. The accuracy of the LLM’s evaluations and the quality of its imagined trajectories are crucial for the robot's success. While larger LLMs show better performance, they come at a higher computational cost. Future work aims to refine the feedback process, explore multimodal feedback incorporating images and videos, and adapt the method for more complex real-world scenarios. The vision is a future where robots learn intuitively by understanding our intentions, eliminating the tedious need for reward engineering.