This project provides a guide to fine-tuning the OpenCodeInterpreter-DS-6.7B coding LLM model for text-to-SQL code generation using the QLoRA+ technique. QLoRA+ is an improvement over the standard LoRA (Low-Rank Adaptation) approach that allows for different learning rates for the adapter matrices, significantly reducing the number of trainable parameters while maintaining model performance and speeding up fine-tuning by up to 2x. The fine-tuned model can generate accurate SQL queries based on natural language questions and database schemas. A Gradio app is created to showcase the model's capabilities, allowing users to interact with it in real-time by providing a schema and asking questions
-
Notifications
You must be signed in to change notification settings - Fork 0
Fine-tuning coding LLM OpenCodeInterpreter-DS-6.7B for Text-to-SQL Code Generation on a Single A100 GPU in PyTorch
License
jordandeklerk/OpenCodeInterpreter-Finetune-SQL
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
Fine-tuning coding LLM OpenCodeInterpreter-DS-6.7B for Text-to-SQL Code Generation on a Single A100 GPU in PyTorch
Topics
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published