Large language models (LLMs) have revolutionized how we interact with technology, but they still struggle with generating highly structured outputs like code, formulas, or markup. Think of it like a brilliant improvisational artist who can riff endlessly but struggles to compose a structured sonata. They have the creativity, but lack the precision. Current methods for enforcing structure, like grammar-constrained decoding (GCD), act like a strict conductor, limiting the artist’s choices to ensure grammatical correctness. However, this rigid approach can stifle the LLM’s creativity, leading to outputs that are technically correct but lack the natural flow and richness of the model's original distribution. Imagine forcing the artist to play only certain notes – the result might be grammatical, but dull. This paper introduces a groundbreaking new concept called grammar-aligned decoding (GAD). Instead of a rigid conductor, GAD acts like a collaborative coach, guiding the LLM to explore its creative potential within the boundaries of grammatical correctness. The proposed algorithm, Adaptive Sampling with Approximate Expected Futures (ASAp), learns from the LLM’s attempts, iteratively refining its guidance to better align with the model’s inherent creativity. It's like the coach observing the artist's performance and providing feedback to help them improve while staying within the rules of composition. The results are impressive: ASAp generates outputs that are not only grammatically correct but also more likely according to the LLM’s original distribution, meaning they are more natural, richer, and better reflect the model’s true capabilities. This research opens exciting new avenues for LLM applications. By aligning grammar and meaning, we can unlock the full potential of these powerful models, enabling them to generate complex, structured outputs that are both creative and correct. While ASAp represents a significant step forward, the journey towards perfect alignment is ongoing. Future research will focus on improving the algorithm’s efficiency and exploring new techniques for even faster convergence. This is just the beginning of a new era for LLMs, where structure and creativity work hand in hand to achieve remarkable results.