Large language models (LLMs) are revolutionizing how we interact with technology, but their massive size presents a significant hurdle for deployment. Optimizing these behemoths for speed without sacrificing performance is a constant challenge. A new quantization technique called "Integer Scale" offers a potential breakthrough, promising faster LLM inference with minimal impact on accuracy. The core problem lies in the computational bottleneck of fine-grained quantization methods. While these methods offer high accuracy by quantizing weights and activations in smaller groups, the constant conversion between integer and floating-point representations during computation slows things down. Integer Scale addresses this by using integer scaling factors, eliminating the need for costly type conversions. This seemingly simple change leads to significant speed improvements, up to 1.85x faster than traditional floating-point quantization and even outperforming some state-of-the-art 4-bit quantization methods. The benefits extend to complex model architectures like Mixture-of-Experts and the challenging-to-quantize LLaMA-3, where Integer Scale demonstrates both speed and accuracy gains. This technique is a "free lunch" because it doesn't require extra training or calibration, making it a simple plug-and-play solution for existing quantization methods like GPTQ, AWQ, and OmniQuant. The results on standard benchmarks like C4, LAMBADA, and common sense reasoning tasks show that Integer Scale maintains accuracy while significantly boosting inference speed. This opens doors to wider LLM deployment, enabling faster and more efficient AI applications on various devices. While Integer Scale offers a promising path to faster LLM inference, challenges remain, particularly the risk of integer overflow. Future research could explore mitigation strategies to further enhance the robustness and applicability of this exciting new technique.