Large language models (LLMs) are impressive, but they can accidentally leak private info from their training data. Researchers are constantly working on ways to keep this data safe. One method is differential privacy (DP), which adds noise during training to mask individual data points. However, this can be computationally expensive and sometimes makes the model less accurate. A newer approach, called Private Mixing of Ensemble Distributions (PMixED), offers better accuracy by mixing outputs from multiple private models with a public one. But PMixED has a fixed privacy level, meaning it can only handle a certain number of requests before its privacy guarantees weaken. This makes it tough to use in real-world situations where you don’t know how many requests you’ll get. A team at USC has improved PMixED with a new technique called Adaptive PMixED (AdaPMixED). AdaPMixED adjusts its privacy protection on the fly, depending on the query it receives. It does this using two clever tricks: noisy screening and data-dependent analysis. Noisy screening filters out queries that are likely to cause a privacy breach, and data-dependent analysis tailors the privacy protection to the specific data being processed. The result is a system that's both more private and more accurate. Tests on several datasets, including WikiText-103 and One Billion Word, showed that AdaPMixED could handle 100,000 queries with a reasonable privacy loss while being more accurate than traditional DP methods. This is a big step forward, as previous methods struggled to handle so many queries while maintaining good performance. While more work is needed, AdaPMixED shows that private prediction can be a practical way to protect user data in large-scale LLM applications.