Navigating the sea of online health information can feel overwhelming. Conflicting advice, questionable sources, and complex medical jargon make it tough to find trustworthy answers. But what if AI could help? A new research paper explores how well current AI models can answer consumer health questions, revealing some intriguing insights into the future of AI-powered health advice. Researchers investigated a method to evaluate the quality of answers generated by large language models (LLMs). The challenge? Open-ended health questions require nuanced answers, making traditional AI evaluation methods like simple text matching insufficient. Experts are the gold standard, but their time is precious. The study introduced a new approach: ranking AI-generated answers alongside human-written web documents using a sophisticated retrieval model. This method, called Normalized Rank Position (NRP), assesses how well AI answers stack up against trusted sources without needing constant expert input. This allows for scalable evaluation of various AI models and prompting techniques. One key finding highlights the critical role of prompting—how the question is posed to the AI. Carefully crafted prompts significantly boosted the quality of answers, especially for larger, more complex models. Even models trained on instructions benefitted from specialized health-related prompts. Model size also mattered. Larger models generally performed better, but the study also found that larger isn't always better. There seems to be a point of diminishing returns, where adding more parameters doesn’t necessarily lead to significantly better answers. Finally, comparing the AI's ranking with an expert's judgment revealed a high degree of agreement, suggesting NRP is a reliable way to assess the quality of AI-generated health advice. While this research is promising, it also acknowledges limitations. Future research will delve into more complex datasets and explore how to evaluate AI systems that can cite their sources, mimicking how humans explain and justify their reasoning. This study offers a glimpse into how we might one day leverage AI to provide reliable and accessible health information to everyone, paving the way for a future where AI empowers us to make informed decisions about our health.