Large language models (LLMs) are impressive, but customizing them for specific tasks can be slow. Dynamic adapters, like using Low-Rank Adaptation (LoRA) with Mixture-of-Experts (MoE), offer a way to make LLMs more adaptable without extensive retraining. However, these adapters can surprisingly slow down the LLM's speed by a significant amount, sometimes more than doubling the time it takes to generate text. This slowdown happens because the way dynamic adapters are currently designed leads to many small, inefficient operations on the GPU, especially during the text generation phase. Researchers have discovered that these fragmented operations are the main bottleneck. To address this, a new approach called LoRA-Switch has been developed. Instead of applying adapters at the layer or block level, LoRA-Switch uses a token-wise routing mechanism. This means the adapters are chosen for each individual token *before* text generation begins. This allows for a clever optimization: the chosen adapters can be merged directly into the LLM's core calculations, significantly reducing the overhead. Furthermore, a specialized CUDA kernel called SGMM has been designed to handle this merging process very efficiently. The result? LoRA-Switch achieves similar accuracy improvements as other dynamic adapters but is much faster. In tests, it reduced the text generation latency by more than 2.4 times compared to other methods, bringing the speed much closer to the original LLM without adapters. This breakthrough is a big step towards making LLMs more efficient and practical for real-world applications. While the initial results are promising, future research will likely focus on further optimizing the prefilling phase (the initial processing of text before generation) and exploring how these techniques can be applied to even larger and more complex LLMs.