Galactica-1.3B
| Property | Value |
|---|---|
| Model Size | 1.3B parameters |
| License | CC BY-NC 4.0 |
| Release Date | November 2022 |
| Author | Meta AI |
| Paper | View Paper |
What is galactica-1.3b?
Galactica-1.3B is a specialized language model developed by Meta AI's Papers with Code team, designed specifically for scientific tasks. It's part of the Galactica model family, trained on a massive corpus of 106 billion tokens of scientific text, including papers, textbooks, and scientific websites. This base version represents a balanced combination of capability and efficiency.
Implementation Details
The model utilizes a decoder-only Transformer architecture with specific modifications optimized for scientific content processing. It can be deployed using PyTorch and supports various precision formats including FP16 and INT8 for efficient inference.
- Supports both CPU and GPU inference with optimized implementations
- Compatible with the Hugging Face transformers library
- Offers flexible deployment options with different precision levels
- Includes specialized tokenization for scientific content
Core Capabilities
- Citation prediction and reference handling
- Scientific question answering
- Mathematical reasoning
- Document summarization
- Molecular property prediction
- Entity extraction from scientific text
Frequently Asked Questions
Q: What makes this model unique?
The model's distinctive feature is its specialized training on scientific content, making it particularly effective for academic and research-related tasks. Unlike general-purpose language models, it understands scientific notation, citations, and technical concepts.
Q: What are the recommended use cases?
The model is primarily intended for researchers studying language models in scientific domains and developers building scientific tools. However, it's important to note that production use should include safeguards due to potential hallucination risks.
