Typhoon-7B
| Property | Value |
|---|---|
| Parameter Count | 7.27B |
| License | Apache-2.0 |
| Primary Languages | Thai, English |
| Paper | View Paper |
| Tensor Type | BF16 |
What is Typhoon-7B?
Typhoon-7B is a groundbreaking pretrained Thai language model developed by SCB 10X AI Team. Built on the Mistral-7B architecture, it represents a significant advancement in Thai language processing, offering performance comparable to GPT-3.5 while being 2.62 times more efficient in Thai text tokenization.
Implementation Details
The model is implemented using the Transformers library and features a decoder-only architecture with 7.27 billion parameters. It utilizes BF16 tensor types for optimal performance and requires Transformers 4.34.0 or newer for operation.
- Architecture: Decoder-only transformer model based on Mistral-7B
- Benchmark Performance: Leads in Thai examination benchmarks including ONET, IC, TGAT, TPAT-1, and A-Level
- Token Efficiency: 2.62x more efficient in Thai text tokenization compared to GPT-3.5
Core Capabilities
- State-of-the-art performance in Thai language understanding and generation
- Dual language proficiency in Thai and English
- Superior performance in academic benchmarks, surpassing human averages in several metrics
- Commercial-friendly Apache-2.0 license
Frequently Asked Questions
Q: What makes this model unique?
Typhoon-7B stands out for its unprecedented performance in Thai language tasks, outperforming all existing open-source Thai language models while maintaining efficiency in tokenization. It's particularly notable for achieving scores above average human performance in several Thai examination benchmarks.
Q: What are the recommended use cases?
As a pretrained base model, it's best suited for Thai language processing tasks with one/few-shot learning applications. Note that it's not instruction-tuned, so direct instruction following may require additional fine-tuning. Users should be aware that the model doesn't include content moderation mechanisms.
