| Property | Value |
|---|---|
| License | MIT |
| Framework | PyTorch |
| Base Model | xlm-roberta-base |
| F1 Score | 0.8489 |
What is xlm-roberta-base-finetuned-panx-all?
This model is a specialized version of XLM-RoBERTa fine-tuned for multilingual Named Entity Recognition (NER) using the PAN-X dataset. Developed as part of the "NLP with Transformers" book, it demonstrates impressive performance with an F1 score of 84.89% and accuracy of 84.32% on token classification tasks.
Implementation Details
The model was trained using carefully selected hyperparameters, including a learning rate of 5e-05, batch size of 24, and Adam optimizer. The training process spanned 3 epochs, showing consistent improvement in performance from an initial validation loss of 0.1883 to a final loss of 0.1739.
- Training conducted with PyTorch 1.9.1
- Implements Transformers 4.12.0
- Utilizes linear learning rate scheduler
- Optimized with Adam optimizer (betas=0.9,0.999)
Core Capabilities
- Multilingual Named Entity Recognition
- Token Classification across languages
- High precision (84.10%) and recall (85.69%)
- Robust performance across different languages in the PAN-X dataset
Frequently Asked Questions
Q: What makes this model unique?
The model combines XLM-RoBERTa's multilingual capabilities with specific optimization for NER tasks, achieving balanced precision and recall metrics. Its training process is well-documented and reproducible, making it ideal for both research and production environments.
Q: What are the recommended use cases?
This model is particularly suited for multilingual named entity recognition tasks, especially when working with the PAN-X dataset or similar multilingual contexts. It's optimal for applications requiring robust cross-lingual entity detection and classification.