Large language models (LLMs) are revolutionizing coding, but they're not perfect. While they can generate impressive chunks of code from simple prompts, these AI assistants can still make mistakes, sometimes producing code that runs without errors but gives the wrong results. A new study dives deep into these non-syntactic errors, uncovering seven distinct types of mistakes, four of which haven't been identified before. These errors range from misinterpreting the instructions in the prompt to overlooking crucial edge cases and even misusing common library functions due to quirks in their training. The study reveals that even state-of-the-art LLMs like GPT-4 and Gemini can fall prey to these issues, particularly when dealing with complex coding problems or those requiring knowledge of external dependencies. Researchers discovered that the way a coding problem is phrased and structured significantly impacts the LLM's understanding and the code it generates. Subtle ambiguities in wording or the placement of key instructions can lead the AI astray. Moreover, the provided input-output examples in the prompt can either guide the LLM correctly or bias it toward incorrect solutions if they're not comprehensive enough. Perhaps surprisingly, the researchers also found evidence that LLMs sometimes get confused by the subtle differences in how similarly named functions work across different programming languages like Python and Java, suggesting gaps in their training data. The study also explored whether LLMs could detect these mistakes themselves. While GPT-4 showed promise in identifying errors in simpler code, its performance dipped when faced with code involving external libraries. This highlights the need for careful human review of LLM-generated code. To understand *why* these mistakes happen, the researchers went beyond just identifying error types. They pinpointed six root causes, from misleading instructions in the prompt to incorrect knowledge embedded within the LLM itself. They even developed a benchmark of coding problems specifically designed to trigger these errors and tested various methods for automatically identifying the root causes. While current methods aren't perfect, the research lays the groundwork for more robust error detection and correction tools for LLM-generated code. This research has important implications for developers and researchers alike. Developers should be aware of these potential pitfalls and not blindly trust AI-generated code. Researchers, meanwhile, have a roadmap for improving the accuracy and reliability of LLMs for code generation, potentially leading to even more powerful AI coding assistants in the future.