MobileBERT-Uncased-SQuAD-v2
| Property | Value |
|---|---|
| Parameter Count | 24.6M |
| License | MIT |
| Paper | Research Paper |
| Author | csarron |
| Model Type | Question Answering |
What is mobilebert-uncased-squad-v2?
MobileBERT-Uncased-SQuAD-v2 is a lightweight question-answering model that represents a significant achievement in making BERT-like models more accessible for mobile and resource-constrained environments. It's a compressed version of BERT_LARGE that maintains impressive performance while significantly reducing the model size through careful architectural optimizations.
Implementation Details
The model was fine-tuned on the SQuAD 2.0 dataset, containing 130k training samples and 12.3k evaluation samples. It achieves an Exact Match (EM) score of 75.2% and an F1 score of 78.8%, approaching the performance of the original MobileBERT paper which reported 76.2% EM and 79.2% F1 scores.
- Trained using Python 3.7.5 on dual GTX 1070 GPUs
- Uses bottleneck structures for efficiency
- Optimized balance between self-attention and feed-forward networks
- Training completed in approximately 3.5 hours
Core Capabilities
- Efficient question answering on general text
- Handles unanswerable questions (SQuAD v2.0 feature)
- Optimized for mobile deployment
- Supports batch processing with configurable sequence lengths
Frequently Asked Questions
Q: What makes this model unique?
This model stands out for its efficient architecture that achieves near BERT-level performance with only 24.6M parameters, making it suitable for mobile and edge devices while maintaining strong accuracy on question-answering tasks.
Q: What are the recommended use cases?
The model is ideal for mobile applications requiring question-answering capabilities, edge devices with limited resources, and scenarios where quick inference time is crucial while maintaining reasonable accuracy.
