Qwen-LLaMAfied-7B-Chat
| Property | Value |
|---|---|
| License | GPL-3.0 |
| Languages | English, Chinese |
| Framework | PyTorch |
| Model Size | 7B parameters |
What is Qwen-LLaMAfied-7B-Chat?
Qwen-LLaMAfied-7B-Chat is a reconstructed version of the original Qwen-7B-Chat model, specifically calibrated to align with the LLaMA/LLaMA-2 architecture. This white-labeled version maintains the original model's capabilities while offering improved compatibility with LLaMA-based infrastructures.
Implementation Details
The model utilizes LlamaForCausalLM for inference and employs a GPT2Tokenizer converted from the original tiktoken. It follows the ChatML prompt format and has undergone numerical weight alignment and preliminary reinforcement learning to maintain consistency with the original model.
- MMLU Score: 53.48 (only 0.42 points below original)
- CEval Score: 54.13 (only 0.05 points below original)
- Supports both English and Chinese languages
- White-labeled implementation
Core Capabilities
- Multi-language support (English and Chinese)
- Strong performance across various domains (Social Science: 66.19%, Humanities: 58.76%)
- Compatible with LLaMA/LLaMA-2 infrastructure
- Maintains original model capabilities with minimal performance loss
Frequently Asked Questions
Q: What makes this model unique?
This model uniquely combines the capabilities of Qwen-7B-Chat with LLaMA architecture compatibility, making it more versatile for existing LLaMA-based infrastructures while maintaining nearly equivalent performance metrics.
Q: What are the recommended use cases?
The model is particularly well-suited for bilingual applications, general text generation, and tasks requiring strong performance in social sciences and humanities. It's ideal for users who need LLaMA compatibility while maintaining Qwen's capabilities.
