| Property | Value |
|---|---|
| Parameter Count | 13 Billion |
| Context Length | 8192 tokens |
| Training Tokens | 1.5T (1.32T English, 0.18T Japanese) |
| License | Apache License 2.0 |
| Developer | Preferred Networks, Inc |
What is plamo-13b-instruct?
PLaMo-13B-Instruct is an advanced bilingual language model specifically fine-tuned for instruction-following tasks in both Japanese and English. Built upon the PLaMo-13B base model, it represents a significant achievement in creating a capable bilingual AI system with extended context handling capabilities.
Implementation Details
The model utilizes a causal decoder-only architecture and implements a sentencepiece tokenizer trained on carefully curated pretraining datasets. With its 8192 token context window, it offers substantial capacity for handling longer conversations and complex tasks.
- Extensive training on 1.5T tokens across English and Japanese
- Fine-tuned on multiple high-quality Japanese datasets including translated versions of dolly-15k and Anthropic HH-RLHF
- Implements advanced sampling parameters with temperature control and top-p/top-k filtering
Core Capabilities
- Bilingual instruction following in Japanese and English
- Extended context handling with 8192 token window
- Advanced text generation with configurable sampling parameters
- Robust performance across multiple task types
Frequently Asked Questions
Q: What makes this model unique?
PLaMo-13B-Instruct stands out for its strong bilingual capabilities and extensive fine-tuning on Japanese datasets, making it particularly effective for Japanese language tasks while maintaining English proficiency. The extended context length of 8192 tokens is another distinguishing feature.
Q: What are the recommended use cases?
The model is well-suited for bilingual applications requiring Japanese and English language processing, instruction-following tasks, and scenarios benefiting from extended context understanding. It's particularly valuable for applications requiring nuanced understanding of Japanese language and culture.