RakutenAI-7B-chat
| Property | Value |
|---|---|
| Parameter Count | 7.37B |
| License | Apache 2.0 |
| Research Paper | arXiv:2403.15484 |
| Tensor Type | BF16 |
| Languages | Japanese, English |
What is RakutenAI-7B-chat?
RakutenAI-7B-chat is a sophisticated bilingual language model developed by Rakuten Group, Inc. that achieves state-of-the-art performance in Japanese language understanding while maintaining strong English capabilities. Built on the Mistral-7B architecture, it features an expanded vocabulary of 48k tokens (up from 32k) to better handle Japanese character encoding.
Implementation Details
The model leverages the Mistral model architecture and is based on the Mistral-7B-v0.1 pre-trained checkpoint, demonstrating successful weight retrofitting. It has been fine-tuned using a diverse mix of datasets including JSNLI, RTE, KUCI, BELEBELE, and others.
- Advanced tokenization with 48k vocabulary size
- Optimized for both Japanese and English processing
- Built on proven Mistral architecture
- Implements efficient BF16 tensor operations
Core Capabilities
- Top performance on Japanese language understanding benchmarks
- Competitive performance on English test sets
- Efficient bilingual processing
- Highest performance among Open LLMs of similar size (0.393 score on Nejumi LLM リーダーボード Neo)
Frequently Asked Questions
Q: What makes this model unique?
The model's distinctive feature is its optimized performance for Japanese language processing while maintaining strong English capabilities, achieved through an expanded vocabulary and careful architecture design based on Mistral-7B.
Q: What are the recommended use cases?
The model is particularly well-suited for bilingual applications requiring Japanese and English language understanding, conversational AI, and general text generation tasks in both languages.
