fairlex-scotus-minilm
| Property | Value |
|---|---|
| License | CC-BY-NC-SA-4.0 |
| Language | English |
| Task | Fill-Mask |
| Framework | PyTorch |
What is fairlex-scotus-minilm?
fairlex-scotus-minilm is a specialized legal language model that's part of the FairLex benchmark suite, specifically designed for processing Supreme Court of the United States (SCOTUS) documents. It's a compact yet powerful model featuring 6 Transformer blocks, 384 hidden units, and 12 attention heads, warm-started from MiniLMv2 using a distilled version of RoBERTa.
Implementation Details
The model utilizes a mini-sized BERT architecture optimized for legal text processing. It's implemented using the Transformers library and PyTorch framework, making it easily accessible for developers and researchers. The model was developed by the CoAStaL NLP Group and is specifically trained on SCOTUS corpora to ensure domain-specific understanding.
- 6 Transformer blocks for efficient processing
- 384 hidden units for representation learning
- 12 attention heads for multi-dimensional attention
- Warm-started from MiniLMv2 with RoBERTa distillation
Core Capabilities
- Fill-mask task specialization for legal text
- Domain-specific understanding of Supreme Court documents
- Efficient processing with minimal computational requirements
- Multilingual benchmark evaluation capabilities
Frequently Asked Questions
Q: What makes this model unique?
This model is specifically designed for legal text processing with a focus on SCOTUS documents, combining efficiency with domain expertise. Its compact architecture makes it accessible while maintaining performance on legal text tasks.
Q: What are the recommended use cases?
The model is ideal for legal text analysis, particularly for tasks involving Supreme Court documents. It's especially suited for fill-mask tasks, legal research, and document analysis where understanding of legal context is crucial.
