Unit testing is vital in software development, ensuring individual parts of a program work as expected. While automated tools help, they often produce tests that are hard to read and maintain. Large Language Models (LLMs) offer a potential solution, but how good are they at generating reliable tests? This large-scale investigation explored using four different LLMs and five prompting strategies to generate over 200,000 unit tests for hundreds of Java classes. The results were then compared against EvoSuite, a standard automated testing tool. The study looked at whether the LLM-generated tests were correct, readable, and covered the code adequately. It also checked for "test smells," which are signs of poorly designed tests. The findings reveal a nuanced picture: LLMs are capable of creating readable tests, especially when guided by smarter prompts. However, they aren’t as good as established tools at ensuring thorough code coverage. Interestingly, certain prompt strategies significantly improved test quality. More advanced prompt methods led to better tests than just asking LLMs to generate tests directly. Another key takeaway is the prevalence of test smells. While LLMs have the potential to improve automated testing, they currently create tests riddled with "magic numbers" and unclear assertions. This means developers would still need to fix these issues, impacting real-world usability. This study represents a crucial step in understanding how LLMs can help automate testing. It highlights that while LLMs show promise, they’re not yet a perfect replacement for existing tools. Future work should focus on prompt refinement and new test smell reduction techniques to unlock the full power of LLMs in automated testing.