Aligning large language models (LLMs) with human preferences is crucial for creating helpful and harmless AI assistants. However, traditional methods like Reinforcement Learning from Human Feedback (RLHF) and Direct Alignment from Preferences (DAP) often require vast amounts of human input, making the process costly and time-consuming. New research introduces a more sample-efficient approach, called Sample-Efficient Alignment (SEA), to address this bottleneck. By framing LLM alignment as a contextual dueling bandit problem, SEA leverages Thompson sampling, a technique from bandit theory, to strategically select the most informative comparisons for human feedback. This allows the LLM to learn more effectively from fewer examples. SEA maintains an “epistemic reward model” that captures the uncertainty in human preferences, guiding the LLM to explore areas where it’s most unsure. It also uses a “policy-guided search” to efficiently navigate the vast space of possible responses, further enhancing sample efficiency. Experiments show SEA consistently outperforms existing methods, achieving better alignment with fewer human queries across different model sizes and alignment techniques. This breakthrough has significant implications for making LLM alignment more practical and accessible, potentially accelerating the development of advanced, human-centric AI assistants.