Aligning large language models (LLMs) with human intentions is crucial for their safe and effective use. Traditionally, this has involved reinforcement learning from human feedback (RLHF), where humans provide preferences to guide the model's learning. However, this approach has limitations, such as noisy labels, high annotation costs, and privacy concerns. New research introduces a groundbreaking alternative: Alignment from Demonstrations (AfD). Instead of relying on human preferences, AfD leverages readily available, high-quality demonstration data, like expert-written text or transcripts of successful customer service interactions. This approach sidesteps the challenges of RLHF while offering several advantages. First, demonstration data is generally cleaner and less noisy than preference data. Second, it eliminates the need for continuous human input, reducing costs and speeding up the alignment process. Third, AfD doesn't require the assumptions about human preferences that traditional methods often make. Finally, it can be applied locally to private data, addressing privacy concerns. The core idea of AfD is to frame LLM alignment as a sequential decision-making problem, similar to how reinforcement learning works. The research introduces a novel technique called trajectory distribution matching, which aims to make the LLM's output distribution resemble the distribution of the demonstration data. This is achieved by training a reward model that distinguishes between LLM-generated text and demonstration text. This reward model then guides the LLM to produce outputs more aligned with the demonstrations. Experiments on challenging alignment tasks show that AfD can match or even exceed the performance of traditional RLHF methods. This suggests that AfD is a powerful and efficient alternative for aligning LLMs, opening up exciting possibilities for building safer and more reliable AI systems. While promising, AfD also presents new challenges. One key area for future research is understanding how the diversity and quality of demonstration data impact alignment effectiveness. Another challenge is the potential for overoptimization, where the LLM becomes too focused on the specific demonstration data and loses generalizability. Despite these challenges, AfD represents a significant step forward in LLM alignment, offering a practical and promising path towards building more aligned and trustworthy AI.