| Property | Value |
|---|---|
| Parameters | 2.4B |
| License | GML |
| Languages | English, Chinese |
| Framework | PyTorch |
What is MiniCPM-2B-sft-fp32?
MiniCPM-2B-sft-fp32 is an innovative end-side language model developed jointly by ModelBest Inc. and TsinghuaNLP. With just 2.4B parameters (excluding embeddings), it achieves remarkable performance comparable to much larger models like Mistral-7B, particularly excelling in Chinese, Mathematics, and Coding tasks.
Implementation Details
The model is implemented using PyTorch and requires Transformers >= 4.36.0. It's optimized for both performance and efficiency, capable of running on consumer hardware including mobile devices. After Int4 quantization, it can perform streaming output at speeds exceeding human speech.
- Supports efficient fine-tuning on single consumer GPUs (1080/2080)
- Full parameter fine-tuning possible on 3090/4090
- Includes mobile deployment capabilities through MLC-LLM and LLMFarm
Core Capabilities
- Matches or exceeds Mistral-7B performance on general benchmarks
- Superior performance in Chinese, Mathematics, and Coding
- Outperforms Llama2-13B, MPT-30B, and Falcon-40B
- Mobile-ready with Int4 quantization support
- Streaming output capabilities
Frequently Asked Questions
Q: What makes this model unique?
The model's ability to achieve performance comparable to much larger models (7B-70B parameters) while maintaining a compact 2.4B parameter size makes it particularly unique. It's one of the few models optimized for mobile deployment while retaining high performance.
Q: What are the recommended use cases?
The model is ideal for mobile applications, edge computing scenarios, and situations requiring efficient deployment with limited computational resources. It's particularly well-suited for applications requiring Chinese language processing, mathematical computations, and coding tasks.