Large language models (LLMs) are impressive, but they can still make mistakes. How can we tell when they're not sure about their answers? A new research paper introduces "MAQA," a clever way to test how well LLMs quantify their uncertainty. MAQA focuses on questions with multiple valid answers, like "Which countries border the Mediterranean Sea?" This adds an extra layer of complexity, as the LLM needs to not only identify correct answers but also recognize when it might be missing some. The researchers tested various uncertainty quantification methods, including looking at the probabilities assigned to different words (for models where this information is available) and comparing different answers generated by the same model. Interestingly, they found that the probability distributions, especially entropy, are still useful indicators of uncertainty, even with multiple valid answers. However, LLMs tend to be overconfident when tackling reasoning tasks, especially after giving their first answer. This makes judging their uncertainty in these scenarios more difficult. Overall, consistency checks (seeing how similar multiple responses to the same question are) emerged as a strong method for evaluating uncertainty across various tasks, especially when dealing with multiple possible answers. This research is a crucial step towards making LLMs more reliable. By understanding how and when these models are unsure, we can better trust their responses and work towards improving their ability to express uncertainty in the future.