Large language models (LLMs) like Mixtral and Phi-1.5 are revolutionizing how we interact with technology, but their immense size makes them difficult to run on everyday devices. Imagine having the power of these advanced AI models right on your phone or laptop! That’s the promise of Mixture-of-Experts (MoE) models, which cleverly activate only small portions of the model at a time. However, even with this efficiency, MoE models still struggle with memory limitations on edge devices. Enter HOBBIT, a groundbreaking new system that turbocharges MoE inference by dynamically loading and managing experts. Think of it as a highly efficient memory manager for your LLM. HOBBIT's secret sauce is its use of mixed-precision loading. It identifies the most important parts of the model for a given task and prioritizes loading those in high precision. Less critical components are loaded in lower precision, drastically reducing the memory footprint and loading times without significantly sacrificing accuracy. HOBBIT employs a three-pronged approach. First, it dynamically loads experts as needed, using a gating network to determine their importance and choose the appropriate precision. Second, it prefetches experts for upcoming tasks, intelligently predicting what the model will need next. Finally, it employs a multidimensional caching policy that combines various strategies to manage the expert cache efficiently. The results? HOBBIT achieves up to a 9.93x speedup in decoding performance compared to existing methods, and dramatically reduces the initial loading time. This allows for faster and smoother LLM interactions on resource-constrained devices, opening the door to powerful AI applications on your phone, laptop, and beyond. While HOBBIT primarily focuses on GPU-centric computation, it also supports CPU assistance for even greater flexibility. This research demonstrates a significant advancement in deploying large, powerful LLMs on edge devices, bringing us closer to a future where AI is readily accessible everywhere.