| Property | Value |
|---|---|
| Model Size | 70B parameters |
| Developer | Snowflake |
| Model Type | Text-to-SQL |
| Base Architecture | LLaMA 3.1 |
| Paper | ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback |
| Model URL | Hugging Face |
What is Llama-3.1-Arctic-ExCoT-70B?
Llama-3.1-Arctic-ExCoT-70B is a state-of-the-art Text-to-SQL model developed by Snowflake's AI research team. It represents a significant advancement in natural language to SQL translation, achieving an impressive 68.53% execution accuracy on the BIRD test benchmark. The model combines Chain-of-Thought (CoT) prompting with SQL execution-based Direct Preference Optimization (DPO), creating a novel framework that learns from execution results rather than human preferences.
Implementation Details
The model implements the ExCoT framework, which innovatively uses execution results as feedback signals for optimization. This approach eliminates the need for expensive human annotations while maintaining high-quality results. Built on the LLaMA 3.1 70B base model, it achieved a remarkable improvement in execution accuracy from 57.37% to 68.51% on the BIRD-dev set.
- Utilizes public datasets (BIRD and Spider) for training
- Implements execution-based DPO for optimization
- Achieves best-in-class performance in single-model, single-inference category
Core Capabilities
- Superior Text-to-SQL translation accuracy
- Outperforms major models like GPT-4 and Claude 3.5 on BIRD benchmark
- Scalable optimization without human annotation requirements
- Robust execution-guided reasoning capabilities
Frequently Asked Questions
Q: What makes this model unique?
The model's ExCoT framework sets it apart by using SQL execution results as feedback for optimization, rather than relying on human preferences. This innovative approach has led to state-of-the-art performance while maintaining scalability.
Q: What are the recommended use cases?
The model is specifically designed for Text-to-SQL applications, making it ideal for database query generation, natural language interface to databases, and automated SQL query writing tasks where high accuracy is crucial.