The intricate world of patent drafting, traditionally the domain of legal experts, is witnessing a transformative shift with the advent of AI. Imagine software that could automatically generate patent claims, saving inventors time and resources. But how accurate and reliable are these AI-powered tools? Researchers have introduced PatentEval, a new benchmark designed to assess the quality of AI-generated patent text. This benchmark focuses on two key tasks: generating abstracts from claims (claims-to-abstract) and generating subsequent claims based on preceding ones. The study evaluated various language models, ranging from specialized patent-trained models to general-purpose large language models like Llama 2 and Falcon. The results reveal a fascinating dynamic. While AI models show promise, they still struggle with common errors. Surprisingly, larger models don’t always guarantee better results. For instance, some AI-generated abstracts missed crucial details or became overly verbose, while others excelled. In claim generation, challenges arose with maintaining proper formatting, dependency clarity, and avoiding redundancy. Interestingly, ChatGPT showed impressive performance, often matching or even exceeding the quality of human-drafted text in both tasks. However, other models grappled with grammatical errors, irrelevant content, and contradictory information. PatentEval offers a crucial step towards understanding the potential and limitations of AI in patent drafting. It highlights the need for refined evaluation metrics that go beyond simple semantic similarity and delve into the nuances of legal language. While a fully automated patent-writing AI may still be on the horizon, this research signals exciting progress. It suggests that AI could become a powerful assistant for patent professionals, helping to streamline the drafting process and potentially democratizing access to patent protection for inventors.