| Property | Value |
|---|---|
| Parameter Count | 11 Billion |
| Language | Polish (primary), English (secondary) |
| License | Apache 2.0 |
| Developer | SpeakLeash & ACK Cyfronet AGH |
| Model Type | Causal decoder-only |
What is Bielik-11B-v2.3-Instruct?
Bielik-11B-v2.3-Instruct is a state-of-the-art Polish language model that represents a significant advancement in Polish NLP. Created through a linear merge of three previous Bielik versions, it combines sophisticated training techniques with extensive Polish language datasets to deliver exceptional performance in both Polish and English language tasks.
Implementation Details
The model employs several innovative training approaches, including weighted tokens level loss, adaptive learning rate scaling, and masked prompt tokens. It uses ChatML as its prompt format and has been fine-tuned using DPO-Positive methodology with over 66,000 carefully curated examples.
- Trained on 20+ million instructions comprising 10+ billion tokens
- Implements multi-turn conversation capabilities
- Available in multiple quantized versions (GGUF, GPTQ, FP8)
- Uses the ALLaMo framework for efficient training
Core Capabilities
- Achieves 65.71 score on Open PL LLM Leaderboard, outperforming models up to 70B parameters
- Scores 8.556250 on Polish MT-Bench, surpassing GPT-3.5-turbo
- Demonstrates strong emotional intelligence with 70.86 score on Polish EQ-Bench
- Excels in both Polish and English language tasks
Frequently Asked Questions
Q: What makes this model unique?
It's the first Polish language model to achieve competitive performance against much larger models while maintaining excellent Polish language capabilities. It combines efficient architecture with sophisticated training techniques specifically optimized for Polish language understanding.
Q: What are the recommended use cases?
The model excels in Polish language processing tasks including sentiment analysis, categorization, text classification, and general conversation. It's particularly suitable for applications requiring strong Polish language understanding while maintaining good English language capabilities.