| Property | Value |
|---|---|
| Model Type | BERT |
| Parameters | 110M |
| Architecture | 12-layer, 768-hidden, 12-heads |
| Paper | LEGAL-BERT: The Muppets straight out of Law School |
| Author | nlpaueb (AUEB's NLP Group) |
What is bert-base-uncased-echr?
bert-base-uncased-echr is a specialized BERT model pre-trained specifically on European Court of Human Rights (ECHR) case documents. It's part of the LEGAL-BERT family of models designed to enhance legal NLP applications. This variant was trained on 12,554 ECHR cases from the HUDOC repository, making it particularly effective for tasks involving human rights law and ECHR-related legal analysis.
Implementation Details
The model follows BERT's base architecture but is domain-adapted for legal text. It was trained for 1 million steps with batches of 256 sequences (length 512) using an initial learning rate of 1e-4 on Google Cloud TPU v3-8. The training process specifically focused on ECHR cases to optimize the model's understanding of human rights law terminology and concepts.
- Pre-trained on 12,554 ECHR cases from HUDOC repository
- Uses uncased tokenization for text processing
- Implements standard BERT-base architecture (12-layer, 768-hidden, 12-heads)
- Optimized for legal domain tasks, particularly ECHR-related analysis
Core Capabilities
- Excellent performance on ECHR case analysis tasks
- Highly accurate prediction of legal concepts in human rights context
- Superior performance in masked token prediction for legal terminology
- Specialized understanding of ECHR-specific legal language
Frequently Asked Questions
Q: What makes this model unique?
This model's specialization in ECHR cases makes it uniquely suited for human rights law applications. It shows remarkable accuracy in predicting legal concepts, achieving 99% accuracy in predicting terms like "torture" in relevant contexts, significantly outperforming general-purpose BERT models.
Q: What are the recommended use cases?
The model is best suited for: analyzing ECHR case documents, legal document classification in human rights context, extracting legal concepts from human rights texts, and supporting legal research in European human rights law.