Imagine training an AI to write text that's indistinguishable from human-written content, but without revealing the sensitive data it learned from. That's the challenge addressed by researchers exploring "private prediction" for large-scale synthetic text generation. Traditional methods risk leaking private information, but this new approach focuses on safeguarding the *output*—the generated synthetic text—rather than the AI model itself. How does it work? Researchers prompt a pre-trained large language model (LLM) with source data and then add "noise" to the LLM’s predictions, ensuring privacy without sacrificing quality. This innovative technique tackles the problem of limited synthetic data generation in previous private prediction attempts. By using a smarter privacy analysis and a better selection mechanism, researchers have boosted the amount of generated data from a handful of examples to thousands. This opens doors to wider applications of synthetic data. Furthermore, the method uses ‘public predictions’—predictions made without sensitive data—to generate ‘obvious’ tokens, reducing privacy costs. This method is especially effective for structured data like JSON objects. The approach improves performance on downstream tasks, such as classification, and better preserves the structure of generated data. While this approach offers a significant advancement, researchers acknowledge the inherent trade-off between privacy and data utility. The next step is bridging the gap between private prediction and private fine-tuning, which could revolutionize how we generate synthetic text while protecting sensitive information.