Large language models (LLMs) like ChatGPT are impressive, but can they reliably follow specific formatting instructions? New research suggests they still struggle. A team of researchers introduced FORMATBENCH, a benchmark designed to test how well LLMs adhere to formatting rules across a wide range of tasks, from writing poems with specific structures (like acrostics) to generating code in specialized languages like XDL (Chemical Description Language). They found that even the most advanced LLMs often fail to produce outputs that perfectly match the requested format. This isn't just about aesthetics—in tasks like code generation or controlling autonomous robots, incorrect formatting can lead to system crashes or unexpected behaviors. To address this, the researchers developed a novel technique called "Reinforcing Format Faithfulness" (REFF). REFF leverages the fact that format checking is a decidable problem—you can definitively determine if an output meets the format requirements or not. It uses a "format checker" to evaluate LLM outputs and then uses reinforcement learning to reward the model for generating correctly formatted text. The results are striking: REFF significantly improves the format faithfulness of LLMs without sacrificing the overall quality of the generated content. In some cases, it even surpasses the performance of leading commercial models like GPT-3.5. This research highlights a critical challenge in current LLM development: the gap between understanding instructions and executing them flawlessly. While LLMs excel at generating human-like text, they still struggle with the precision required for tasks with strict formatting needs. REFF offers a promising path towards bridging this gap, potentially leading to more reliable and trustworthy AI systems for a variety of applications.