Evaluating how well large language models (LLMs) can generate code is a complex challenge. Existing benchmarks often fall short because they don't reflect the messy reality of real-world codebases. A new benchmark called DevEval aims to bridge this gap by providing a more realistic testing ground. DevEval stands out because it's built with real-world repositories in mind. It considers not just isolated functions, but also how code interacts within a larger project. This includes dependencies between files and classes, which are crucial in real-world development. The benchmark was carefully crafted by developers who annotated code samples with detailed requirements, original repositories, reference code, and dependencies. This rich context helps LLMs understand the task and generate more relevant code. Initial tests with popular LLMs like GPT-4 and others show promising but imperfect results. While LLMs can generate functionally correct code in some cases, they still struggle with understanding complex dependencies and the nuances of real-world codebases. DevEval's focus on real-world code makes it a valuable tool for evaluating and improving the coding abilities of LLMs. It highlights the need for LLMs to better understand context and dependencies, paving the way for more effective AI-powered coding tools in the future.