MiniLM-L6-Keyword-Extraction
| Property | Value |
|---|---|
| License | Other |
| Framework | PyTorch, Sentence-Transformers |
| Language | English |
| Vector Dimensions | 384 |
What is MiniLM-L6-Keyword-Extraction?
MiniLM-L6-Keyword-Extraction is a powerful sentence embedding model that converts text into dense 384-dimensional vectors. Built on the efficient MiniLM architecture, it's specifically designed for semantic search, clustering, and similarity tasks. The model was fine-tuned on an impressive dataset of over 1 billion sentence pairs, making it particularly robust for real-world applications.
Implementation Details
The model utilizes the sentence-transformers framework and can be easily implemented using either the high-level sentence-transformers API or the lower-level Hugging Face transformers library. It employs mean pooling strategy on token embeddings and includes automatic normalization of the output vectors.
- Pre-trained on nreimers/MiniLM-L6-H384-uncased base model
- Fine-tuned using contrastive learning on diverse datasets
- Trained for 100k steps with a batch size of 1024
- Supports maximum sequence length of 256 tokens
Core Capabilities
- Semantic text embedding generation
- Sentence similarity computation
- Clustering of text documents
- Information retrieval tasks
- Cross-encoder applications
Frequently Asked Questions
Q: What makes this model unique?
The model stands out due to its extensive training on over 1 billion sentence pairs from diverse sources including Reddit comments, scientific papers, and question-answer pairs. This broad training makes it particularly robust for general-purpose sentence embedding tasks.
Q: What are the recommended use cases?
The model excels in semantic search applications, document clustering, similarity comparison between sentences, and as a feature extractor for downstream NLP tasks. It's particularly suitable for applications requiring efficient text representation in a fixed-dimensional space.
