Imagine teaching a robot to perform complex tasks, like preparing a meal or tidying up a room, without constant supervision or real-world practice. This is the promise of offline reinforcement learning (RL). However, traditional offline RL struggles when feedback is sparse—like only knowing whether the robot ultimately succeeded or failed. New research explores how large language models (LLMs) can help. LLMs, like those powering ChatGPT, are great at understanding and generating human language. This research uses them not as robot brains, but as sophisticated reward generators. By analyzing robot actions within a training dataset, the LLM assigns intermediate rewards, providing richer feedback than just a final success/failure signal. This transforms sparse data into dense, actionable learning signals for the robot. However, getting LLMs to assign grounded, relevant rewards is tricky. They can be easily misled by irrelevant details or their own biases. To overcome this, the researchers developed COREN, a consistency-guided reward ensemble framework. COREN uses a clever two-stage process. First, it asks the LLM for multiple reward estimations, each considering different aspects of consistency—like whether rewards make sense within the context of the entire task, the layout of the environment, and the order of actions. Second, it blends these different estimations based on whether the robot ultimately succeeded in the training data. This creates a more refined, environment-specific reward signal. Experiments in a simulated home environment show that robots trained with COREN outperform other offline RL methods and even rival some robots that use LLMs directly for online decision-making. This research opens up exciting new possibilities for training robots more efficiently and safely in complex environments. Imagine robots learning from vast amounts of pre-recorded data without needing costly real-world trials. This could accelerate the development of household robots, assistants for the elderly, and more.