Gemma 2 Baku 2B Instruct
| Property | Value |
|---|---|
| Parameter Count | 2.61B |
| Model Type | Instruction-tuned Language Model |
| Languages | Japanese, English |
| License | Gemma Terms of Use |
| Base Model | google/gemma-2-2b |
What is gemma-2-baku-2b-it?
Gemma 2 Baku 2B Instruct is an advanced bilingual language model that builds upon the foundation of Gemma 2, specifically designed to handle both Japanese and English language tasks. It represents a sophisticated merge of Google's Gemma architecture with specialized instruction-tuning capabilities, achieved through innovative Chat Vector addition and ORPO (Odds Ratio Preference Optimization) techniques.
Implementation Details
The model features a 26-layer transformer architecture with 2304 hidden dimensions. Its unique implementation involves a chat vector addition process where parameter vectors are calculated by subtracting google/gemma-2-2b from gemma-2-2b-it, then adding this difference to the base model. The model specifically excludes embedding layer modifications during this process to maintain linguistic integrity.
- Utilizes eager attention mechanism for optimal batch inference under bfloat16 precision
- Implements the original google/gemma-2-2b-it tokenizer
- Incorporates ORPO fine-tuning using proprietary datasets
Core Capabilities
- Bilingual instruction following in Japanese and English
- Enhanced performance through Chat Vector technology
- Optimized response generation using ORPO
- Efficient handling of conversational tasks
- Support for both CPU and GPU inference
Frequently Asked Questions
Q: What makes this model unique?
This model stands out due to its innovative combination of Chat Vector technology and ORPO optimization, specifically designed for bilingual capability while maintaining the robust performance of the Gemma 2 architecture. The careful parameter merging process ensures optimal performance in both Japanese and English contexts.
Q: What are the recommended use cases?
The model is particularly well-suited for bilingual applications requiring instruction following, conversational AI implementations, and text generation tasks in both Japanese and English. It's optimized for scenarios requiring nuanced understanding and generation in either language.
