MyanBERTa
| Property | Value |
|---|---|
| License | Apache-2.0 |
| Training Data | MyCorpus + Web (5.9M sentences) |
| Training Steps | 528K steps |
| Vocabulary Size | 30,522 subword units |
| Paper | Research Paper |
What is MyanBERTa?
MyanBERTa is a specialized pre-trained language model designed specifically for the Myanmar (Burmese) language. Built on the robust BERT architecture, it represents a significant advancement in Myanmar natural language processing. The model was developed by UCSYNLP and trained on a comprehensive dataset of word-segmented Myanmar text.
Implementation Details
The model employs a sophisticated byte-level BPE tokenization approach, applied after word segmentation, utilizing a vocabulary of 30,522 subword units. The training process involved 528K steps on a substantial dataset comprising 5,992,299 sentences, equivalent to approximately 136 million words.
- Utilizes RoBERTa architecture adapted for Myanmar language
- Implements byte-level BPE tokenization
- Trained on a combination of MyCorpus and Web datasets
- Incorporates word segmentation preprocessing
Core Capabilities
- Fill-mask task prediction
- Myanmar language understanding and processing
- Support for downstream NLP tasks
- Compatible with PyTorch framework
- Inference endpoint availability
Frequently Asked Questions
Q: What makes this model unique?
MyanBERTa is specifically optimized for the Myanmar language, utilizing a word-segmented approach before tokenization, which is crucial for handling the unique characteristics of the Myanmar script and language structure.
Q: What are the recommended use cases?
The model is particularly well-suited for Myanmar language processing tasks, including text classification, named entity recognition, and fill-mask predictions. It serves as a foundation model for various downstream NLP applications in the Myanmar language.
