Large language models (LLMs) are impressive, but their size presents real challenges for accessibility and efficiency. Researchers are constantly looking for ways to shrink these models without sacrificing performance, a process called 'pruning.' A common approach involves splitting the model into smaller parts, pruning each individually, and then stitching them back together. However, this can lead to errors as the pruned sub-models struggle to reconstruct the original model's output. This research explores new techniques to minimize these reconstruction errors, finding that strategies like Block-wise Reconstruction (BR) and Global Propagation (GP) significantly improve the accuracy of the pruned models. But, and here's the twist, simply minimizing these errors isn't always the best approach. Over-optimizing for reconstruction can actually lead to overfitting, where the pruned model performs well on the small set of test data used for pruning but poorly on real-world tasks. The researchers found that larger models are particularly vulnerable to this overfitting problem. One promising solution they explored involves using the LLM itself to generate larger, more representative datasets for pruning. This 'self-generation' technique helps the pruned model generalize better to unseen data, reducing the negative effects of overfitting. This work suggests that the current methods for pruning LLMs need a rethink. While minimizing reconstruction error is important, it's not the whole story. The size and quality of the test data play a crucial role, and techniques like self-generation might be key to unlocking the full potential of smaller, more efficient LLMs.