RedPajama-INCITE-7B-Chat
| Property | Value |
|---|---|
| Model Size | 6.9B parameters |
| License | Apache 2.0 |
| Language | English |
| Training Hardware | 8 A100 GPUs |
| Memory Requirements | 16GB GPU (Full) / 12GB (Int8) |
What is RedPajama-INCITE-7B-Chat?
RedPajama-INCITE-7B-Chat is an advanced language model developed by Together Computer in collaboration with leading institutions including ETH DS3Lab, MILA, and Stanford CRFM. It's a chat-optimized version of the base model, fine-tuned specifically on OASST1 and Dolly2 datasets to enhance its conversational capabilities.
Implementation Details
The model implements a sophisticated architecture requiring transformers version 4.25.1 or higher. It supports multiple inference modes including full GPU, int8-quantized, and CPU inference, making it versatile for different hardware configurations. The model was trained using Adam optimizer with a learning rate of 1e-5 across 79M tokens.
- Supports both float16 and int8 quantization for efficient inference
- Implements specific prompt format: "<human>: [Instruction]\n<bot>:"
- Offers flexible deployment options across different hardware configurations
Core Capabilities
- Natural language understanding and generation
- Conversational AI and chat interactions
- Text completion and assistance
- Multiple inference modes for different hardware requirements
Frequently Asked Questions
Q: What makes this model unique?
The model stands out due to its careful fine-tuning on high-quality conversational datasets (OASST1 and Dolly2) while maintaining efficient resource usage through various optimization options. It offers a balance between performance and accessibility.
Q: What are the recommended use cases?
The model is ideal for chatbot applications, content generation, and general language understanding tasks. However, it should not be used for safety-critical applications or making significant impact decisions affecting individuals or society.
