Large language models (LLMs) like ChatGPT have taken the world by storm, demonstrating incredible abilities to generate text, translate languages, and even write different kinds of creative content. But their massive size presents a challenge for practical use. Imagine trying to run these complex models on your phone or a smaller device – it's just not feasible with the current technology. This is where low-bit quantization comes in, a fascinating technique that's shrinking these giant models down to size without sacrificing too much performance. Essentially, it's like converting a high-resolution image to a smaller file size. You lose some detail, but the overall picture remains. This survey paper explores the core concepts behind low-bit quantization, delving into different number formats and how they're used to represent the model's data more efficiently. It also dives into the systems and algorithms used to make these low-bit models run smoothly on various hardware platforms, from powerful servers to everyday devices. The survey reveals how engineers are tackling the unique challenges posed by LLMs, such as dealing with data outliers that can skew the quantization process. Techniques like equivalent transformations and mixed-precision quantization help fine-tune the accuracy of these compressed models. What's really exciting is how this technology is opening doors to running LLMs on smaller devices, bringing the power of AI to a wider range of applications. Imagine having a powerful language model right in your pocket, ready to assist with everyday tasks. While the field is still young, the advances in low-bit quantization are paving the way for a more accessible and efficient AI future. The quest for even smaller and more accurate low-bit LLMs is ongoing, promising exciting new possibilities for AI-powered applications in the years to come.