Imagine someone extracting your private information from an AI’s memory, not through hacking, but by simply asking the right questions. Sounds like science fiction? A new research paper, "PII-Compass: Guiding LLM training data extraction prompts towards the target PII via grounding," reveals how surprisingly easy this could be. The researchers explored how effectively an attacker could extract personal information, like phone numbers, from a large language model (LLM) trained on a dataset of emails. They found that simple, handcrafted prompts like, "What is the phone number of [name]?" are largely ineffective. However, their innovative approach, called PII-Compass, dramatically increased the success rate. The trick? Grounding the LLM by providing a snippet of related information from the training data, which acts like a compass, guiding the model toward the target information. Using this technique, they extracted almost 7% of the phone numbers in the dataset–meaning one in 15 people's phone numbers were vulnerable. While the researchers focused on phone numbers in emails, this technique has alarming implications for broader data privacy. What other information can be easily extracted? As AI models grow larger, this problem intensifies, creating a paradox: more powerful models may become greater privacy risks. The future of AI hinges on finding solutions that balance powerful capabilities with robust privacy protections. The PII-Compass research is a wake-up call – a step toward understanding and mitigating the risks of unintended data leakage in an increasingly AI-driven world.