Large language models (LLMs) are getting smarter, but they're also getting hungrier for memory. Processing longer conversations, code, or documents with LLMs requires a massive amount of memory, especially for the "key-value cache" used in the attention mechanism. This memory demand is a major bottleneck, limiting the performance and scalability of LLMs. Enter LoL-PIM, a novel hardware-software approach designed to supercharge long-context LLM processing. Current systems struggle with the memory demands of longer contexts, leading to inefficient use of GPUs and high costs. LoL-PIM tackles this head-on by shifting computation directly into the memory itself using Processing-in-Memory (PIM) technology. This minimizes the time spent moving data around, a significant performance drain in traditional LLM systems. LoL-PIM isn't just about raw speed. It introduces a smart partitioning scheme called Intra-module Token-parallel Partitioning (ITPP) that better distributes the LLM's workload across multiple PIM modules. Combined with dynamic memory management, LoL-PIM avoids wasting precious memory on inactive contexts, allowing for larger batch processing and higher throughput. Additionally, a technique called I/O-aware buffering further optimizes data transfer within the PIM system, hiding the latency of input and output operations. The results are impressive. LoL-PIM significantly outperforms both multi-GPU and GPU-PIM systems, offering up to 8.54x and 16.0x speedups, respectively. This boost in performance unlocks the potential for LLMs to engage in even longer, more nuanced conversations, process extensive documents, and analyze complex codebases more efficiently. LoL-PIM opens up exciting new possibilities for deploying LLMs in real-world applications, paving the way for more powerful and responsive AI experiences.