StarChat-β
| Property | Value |
|---|---|
| Parameter Count | 15.5B |
| Model Type | GPT-like Coding Assistant |
| License | BigCode Open RAIL-M v1 |
| Base Model | StarCoderPlus |
What is StarChat-β?
StarChat-β is an advanced language model specifically designed as a coding assistant. It's the second model in the StarChat series, built upon StarCoderPlus and fine-tuned using an uncensored variant of the OpenAssistant-Guanaco dataset. The model excels at both programming tasks and natural language interactions, supporting over 80 programming languages.
Implementation Details
The model utilizes BF16 tensor types and implements a ChatML format for message handling. It was trained using specific hyperparameters including a learning rate of 2e-05, with training conducted across 8 GPUs and 6 epochs.
- Trained on uncensored OpenAssistant-Guanaco dataset
- Uses specialized token ID 49155 for turn completion
- Implements temperature-based sampling with top-k and top-p filtering
Core Capabilities
- Multi-language code generation and assistance
- Natural language understanding and response generation
- Code completion and optimization
- Programming problem-solving across diverse languages
- Interactive chat-based coding support
Frequently Asked Questions
Q: What makes this model unique?
StarChat-β stands out for its uncensored training approach, which has led to improved performance on the Open LLM Leaderboard and enhanced coding capabilities. It combines the strength of StarCoderPlus with refined chat abilities.
Q: What are the recommended use cases?
The model is best suited for educational and research purposes in programming contexts. It excels at code generation, debugging, and providing programming explanations, but should be used with caution as it may generate problematic content when prompted.
