KLUE RoBERTa-small
| Property | Value |
|---|---|
| Parameter Count | 68.1M |
| Model Type | Fill-Mask Transformer |
| Paper | arXiv:2105.09680 |
| Downloads | 4,657 |
| Tensor Type | I64, F32 |
What is roberta-small?
KLUE RoBERTa-small is a compact Korean language model developed by the KLUE benchmark team. It's a lightweight variant of the RoBERTa architecture, specifically trained for Korean language understanding tasks. The model utilizes the masked language modeling approach and is optimized for efficient deployment while maintaining strong performance on Korean NLP tasks.
Implementation Details
The model is implemented using PyTorch and supports the Transformers library. A notable implementation detail is that it requires the BertTokenizer instead of RobertaTokenizer, despite being a RoBERTa model. The model uses the [MASK] token for masked language modeling tasks.
- Utilizes SafeTensors format for improved security and loading efficiency
- Compatible with Inference Endpoints for deployment
- Implements the RoBERTa architecture with Korean language optimization
Core Capabilities
- Masked Language Modeling for Korean text
- Korean language understanding and processing
- Efficient inference with compact parameter size
- Integration with modern ML pipelines
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for being a compact, Korean-specific implementation of RoBERTa, offering a good balance between model size and performance. Its integration with the KLUE benchmark makes it particularly valuable for Korean language understanding tasks.
Q: What are the recommended use cases?
The model is ideal for Korean language processing tasks, particularly masked language modeling. It's well-suited for applications requiring Korean text understanding, while maintaining efficiency due to its smaller parameter count.
