| Property | Value |
|---|---|
| Parameter Count | 77M |
| License | Apache 2.0 |
| Architecture | T5-small-lm-adapt |
| Paper Reference | Research Paper |
What is grammar-synthesis-small?
grammar-synthesis-small is a specialized fine-tuned version of Google's T5-small-lm-adapt model, designed specifically for grammar correction tasks. Built on the JFLEG dataset, this model excels at single-shot grammar correction, particularly for text with multiple errors while maintaining semantic integrity of grammatically correct content.
Implementation Details
The model utilizes a transformer-based architecture with 77M parameters, trained using carefully optimized hyperparameters including a learning rate of 0.0004, cosine scheduler, and Adam optimizer. It employs beam search with 8 beams and includes specific parameters like repetition penalty (1.21) and length penalty (1.0) for optimal output generation.
- Built on T5-small-lm-adapt architecture
- Trained on expanded JFLEG dataset
- Implements beam search with 8 beams
- Uses F32 tensor type for computations
Core Capabilities
- Single-shot grammar correction for heavily error-prone text
- ASR (Audio Speech Recognition) output correction
- Chatbot response refinement
- Correction of OCR-generated text
- Handling of "tortured-phrases" in AI-generated content
Frequently Asked Questions
Q: What makes this model unique?
The model's ability to handle heavily error-prone text while preserving semantically correct content sets it apart. It's particularly effective for correcting ASR outputs and AI-generated content without altering the original meaning of properly constructed phrases.
Q: What are the recommended use cases?
The model excels in correcting transcription outputs, refining chatbot responses, fixing OCR-generated text, and improving AI-generated content. It's particularly valuable for applications requiring maintenance of semantic meaning while correcting grammatical errors.