Large language models (LLMs) have revolutionized various NLP tasks. However, they are vulnerable to data contamination, where training data includes test set information, leading to inflated performance metrics. This post breaks down the different types of data contamination that LLMs face and explains why some are so hard to detect. Imagine training an AI to summarize articles only to discover it's simply memorizing and regurgitating summaries it’s already seen. This is the heart of data contamination, where the model learns to perform the evaluation tasks by memorizing specific instances rather than gaining true language understanding. Researchers at Carnegie Mellon University have introduced a taxonomy categorizing different forms of data contamination. They range from 'verbatim' contamination, where entire test sets leak into the training data, to 'noising,' where test examples are subtly modified, making detection even trickier. The study explored the impact of these contaminations on tasks like text summarization and question answering. Using GPT-2 Large, they discovered some contamination can severely impact performance. For instance, exposure to test data, even in disguised forms, often gave models an unfair advantage akin to having in-domain examples. Interestingly, some forms of contamination, like access to only summaries in summarization tasks, didn't significantly boost performance. This implies that the performance gain primarily stems from increased in-domain data rather than test set memorization. Another interesting challenge with contamination is that models tend to perform better when the format of the training data matches that of the test data. This poses challenges for evaluating models since variations in format affect model performance making it hard to assess true understanding versus format exploitation. The research stresses that current decontamination methods largely focus on detecting whole test set leaks, overlooking the more subtle yet impactful instances of 'noisy' contamination. As LLMs grow, this issue only amplifies, leading to an urgent need for more robust detection and mitigation strategies. The future of LLM training and evaluation relies on recognizing that even slightly altered test data can significantly skew results, urging a shift towards more thorough and comprehensive decontamination practices.