Imagine an AI that can not only write code, but truly *understand* it. That's the tantalizing promise of large language models (LLMs) like GPT-4, which are revolutionizing the way we interact with computers. But how do we measure their code comprehension? Existing benchmarks often focus on code generation or mix up semantic reasoning with software engineering tasks. A new research paper introduces "CRQBench," a unique benchmark specifically designed to isolate and assess an LLM's code reasoning capabilities. This benchmark isn’t about getting an AI to churn out lines of code; it’s about probing its deeper understanding of how code works. The researchers pulled 100 real-world C++ code reasoning questions from code review comments, giving the LLM a code snippet and a question related to control flow or value propagation. These questions were then refined using another LLM combined with human inspection, resulting in a focused set of challenges that accurately assess the LLM’s reasoning abilities. The result? When tested with CRQBench, GPT-4 produced correct, contextually-grounded answers for 65 out of 100 questions. Interestingly, it performed slightly better on "equivalence" queries (determining if two code segments behave differently) compared to "value" queries (figuring out the value of a variable). The researchers also found the model sometimes faltered when lacking essential context, like function definitions or variable usage. And gaps in C++ knowledge tripped it up occasionally. The study highlights the importance of context for LLMs and emphasizes the need for more sophisticated benchmarks that evaluate reasoning separate from other coding tasks. CRQBench is not just a test for LLMs; it’s a crucial step towards developing AI that can truly comprehend and reason about code, opening up a world of possibilities for automated code review, debugging, and so much more.