Large language models (LLMs) are impressive, but their massive size makes them expensive and difficult to deploy. Researchers are constantly developing techniques to compress these models, making them more accessible for practical use. But with so many different methods emerging, how do we know which ones work best? A new benchmark called LLMCBench aims to answer that question. It provides a comprehensive evaluation of various compression algorithms, analyzing their impact on performance, efficiency, and even trustworthiness. The benchmark dives into six key areas: how well the compressed models retain their original performance, their ability to generalize to new tasks, the resources required for training and inference, the effectiveness of hardware acceleration, and the critical aspect of trustworthiness. The results reveal that quantization methods, which reduce the precision of numerical representations within the model, generally outperform sparsity techniques, which prune away less important connections. Specifically, weight-only quantization shines in terms of generalization and performance, meaning these slimmed-down models can still handle a variety of tasks effectively. However, if training resources are limited, sparsity methods like Wanda offer a compelling alternative. The benchmark also highlights the importance of hardware acceleration, with INT4 quantization leading to significant speedups. Interestingly, the research shows that a smaller model doesn't necessarily mean a less trustworthy one. Quantization methods actually improved trustworthiness compared to sparsity. LLMCBench provides invaluable insights for developers seeking to deploy efficient and reliable LLMs. As the field of model compression continues to evolve, benchmarks like this will be crucial for guiding future research and unlocking the full potential of these powerful AI models.