Large language models (LLMs) have become incredibly powerful tools, but are they truly robust? Recent research delves into a critical question: How well do LLMs handle unexpected or adversarial inputs? These inputs can range from slightly altered phrasing designed to trick the model to entirely out-of-distribution data from specialized fields like medicine or e-commerce. The study explores this by examining how LLMs perform on benchmark datasets designed to test both adversarial robustness (resistance to manipulation) and out-of-distribution robustness (ability to generalize to new data). Researchers tested three different LLMs—LLaMA2-7b, LLaMA2-13b, and Mixtral-8x7b—and applied two robustness improvement methods: the Analytic Hierarchy Process (AHP) and In-Context Rewriting (ICR). The results revealed a complex relationship between model size, architecture, and robustness. While ICR seemed effective for smaller models like LLaMA2-7b, especially in improving their ability to correctly identify relevant information (recall), AHP worked better with the larger, more complex Mixtral model. Interestingly, simply scaling up the model size didn't guarantee improved robustness. In fact, the study found a surprising negative correlation between adversarial and out-of-distribution robustness for LLaMA2-13b, suggesting that larger models aren’t always better at handling unexpected inputs. In contrast, Mixtral showed a positive correlation, suggesting its unique architecture might offer inherent advantages. This research underscores the need for tailored strategies to improve LLM robustness. It's not enough to just build bigger models; we need to develop methods that specifically address the challenges posed by adversarial attacks and out-of-distribution data. Future research will explore these relationships in even larger models and more diverse datasets, paving the way for more reliable and trustworthy AI systems.