TAPAS Base Model
| Property | Value |
|---|---|
| Author | |
| License | Apache-2.0 |
| Paper | Original Paper |
| Downloads | 42,933 |
What is tapas-base?
TAPAS base is a sophisticated BERT-like transformer model specifically designed for table understanding and reasoning. Developed by Google, it represents a significant advancement in natural language processing for structured data, particularly tables from Wikipedia. The model employs a unique pre-training approach combining masked language modeling (MLM) with intermediate pre-training for enhanced numerical reasoning capabilities.
Implementation Details
The model was trained on 32 Cloud TPU v3 cores for 1,000,000 steps with a maximum sequence length of 512 and batch size of 512. It uses WordPiece tokenization with a 30,000-word vocabulary and processes inputs in the format: [CLS] Sentence [SEP] Flattened table [SEP]. The training utilizes Adam optimizer with a 5e-5 learning rate and 0.01 warmup ratio.
- Relative position embeddings by default
- Optional absolute position embeddings available
- Pre-trained on Wikipedia data
- Dual pre-training objectives: MLM and table entailment
Core Capabilities
- Table parsing and understanding
- Numerical reasoning on tabular data
- Question answering about tables
- Table entailment verification
Frequently Asked Questions
Q: What makes this model unique?
TAPAS base stands out for its specialized ability to understand and reason about tabular data, combining traditional language modeling with numerical reasoning capabilities through its innovative intermediate pre-training approach.
Q: What are the recommended use cases?
The model is primarily intended for fine-tuning on downstream tasks such as table-based question answering, fact verification, and table entailment. It can also be used for extracting hidden representations from table-question pairs.
