Large language models (LLMs) are transforming how we interact with technology, but deploying these powerful AI tools on edge devices like smartphones and laptops presents a significant challenge. LLMs are memory and computationally intensive, often requiring specialized hardware like GPUs that many edge devices lack. But what if we could harness the power of readily available CPUs to run LLMs efficiently on edge? New research introduces T-MAC, a groundbreaking approach that reimagines how CPUs handle the complex math behind LLMs. T-MAC leverages a clever technique called "table lookup" to drastically speed up calculations. Instead of performing numerous multiplications, T-MAC pre-calculates results and stores them in a table. During inference, the system simply looks up the needed values, bypassing computationally intensive steps. This innovative method not only increases speed but also reduces energy consumption, crucial for battery-powered devices. Tests show T-MAC running LLMs on CPUs up to 4 times faster than existing methods, even rivaling GPU performance in some cases. On an Apple M2 Ultra, T-MAC achieves an impressive 71 tokens/second, exceeding the average human reading speed. Even on a less powerful device like the Raspberry Pi 5, T-MAC reaches a respectable 11 tokens/second, demonstrating its adaptability to a range of hardware. This research opens exciting possibilities for deploying LLMs on a wider array of edge devices. Imagine accessing personalized AI assistance, enhanced privacy through on-device processing, and real-time language processing capabilities on your phone or laptop. T-MAC brings these possibilities closer to reality, ushering in a new era for on-device AI experiences. While the research primarily focuses on CPUs, the core idea behind T-MAC could inspire the development of more efficient dedicated hardware for LLMs in the future. As the demand for on-device AI grows, innovations like T-MAC pave the way for more powerful and accessible AI experiences.