RAG-Specialized-LLM
| Property | Value |
|---|---|
| Base Model | Qwen2.5 14B |
| Training Infrastructure | 8x H100 (80GB) GPUs |
| Developer | Surromind |
| Model URL | https://huggingface.co/Surromind/RAG-Specialized-LLM |
What is RAG-Specialized-LLM?
RAG-Specialized-LLM is a fine-tuned language model specifically optimized for Retrieval-Augmented Generation (RAG) applications. Built on the Qwen2.5 14B architecture, this model has been extensively trained on specialized RAG datasets, Chain-of-Thought (CoT) datasets, and benchmark datasets to deliver precise, source-attributed responses in a structured JSON format.
Implementation Details
The model implements a sophisticated training approach using full fine-tuning with specific parameters including a learning rate of 5e-06, linear scheduler type, and gradient accumulation steps of 64. Training was conducted on 8 H100 GPUs with 80GB memory each, utilizing mixed precision training with bf16 format.
- Structured JSON output format with related documents, sources, and grounded answers
- Source attribution using custom tags (<co: doc_id>)
- Comprehensive training on multiple AIhub datasets including administrative, news, and financial documents
Core Capabilities
- Precise document retrieval and reference
- Structured response generation with source attribution
- Support for multiple document types including administrative, financial, and legal texts
- Chain-of-Thought reasoning capabilities
- Specialized in Korean language processing
Frequently Asked Questions
Q: What makes this model unique?
The model's distinct feature is its specialized JSON output format that includes both raw answers and source-attributed responses, making it ideal for RAG applications requiring transparent source attribution.
Q: What are the recommended use cases?
The model is particularly well-suited for applications requiring document analysis, information retrieval, and fact-based response generation with source attribution, especially in Korean language contexts.
