Tanuki-8B-dpo-v1.0
| Property | Value |
|---|---|
| Parameter Count | 7.51B |
| License | Apache 2.0 |
| Languages | Japanese, English |
| Training Data | 1.3T tokens |
| Tensor Type | BF16 |
What is Tanuki-8B-dpo-v1.0?
Tanuki-8B-dpo-v1.0 is a sophisticated bilingual language model developed by the GENIAC project at Matsuo Lab. It represents a significant advancement in Japanese-English language modeling, having undergone extensive pre-training on 1.3T tokens and further refined through SFT and DPO (Direct Preference Optimization) for enhanced dialogue capabilities.
Implementation Details
The model is built on the transformer architecture and provides several quantized versions for different deployment scenarios. It utilizes the Japanese Alpaca prompt format and has been specifically optimized for conversational tasks.
- Full model available in BF16 format
- Quantized versions: AWQ 4-bit, GPTQ 4-bit/8-bit, and GGUF
- Implements chat template for structured conversations
- Optimized for both single-turn and multi-turn dialogues
Core Capabilities
- Strong performance in Japanese MT-Bench with 7.24 average score
- Excellent results in humanities (9.1), STEM (9.35), and writing (9.05) tasks
- Supports streaming generation for real-time responses
- Specialized in handling both Japanese and English inputs
Frequently Asked Questions
Q: What makes this model unique?
The model stands out for its extensive pre-training on 1.3T tokens and specific optimization for Japanese-English bilingual capabilities. It achieves impressive scores on various benchmarks, particularly in humanities and STEM fields, while maintaining efficient performance through various quantization options.
Q: What are the recommended use cases?
The model is particularly well-suited for conversational AI applications, academic discourse, and technical writing in both Japanese and English. It performs exceptionally well in humanities and STEM-related tasks, making it valuable for educational and research applications.
