[2304.11015] DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction #546
Labels
Algorithms
Sorting, Learning or Classifying. All algorithms go here.
Code-Interpreter
OpenAI Code-Interpreter
in-context-learning
Examples of few-shot prompts for in-context learning.
MachineLearning
ML Models, Training and Inference
Papers
Research papers
Research
personal research notes for a topic
Sqlite
Sqlite DB and tools
[2304.11015] DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction
DESCRIPTION:
DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction
Mohammadreza Pourreza, Davood Rafiei
There is currently a significant gap between the performance of fine-tuned models and prompting approaches using Large Language Models (LLMs) on the challenging task of text-to-SQL, as evaluated on datasets such as Spider. To improve the performance of LLMs in the reasoning process, we study how decomposing the task into smaller sub-tasks can be effective. In particular, we show that breaking down the generation problem into sub-problems and feeding the solutions of those sub-problems into LLMs can be an effective approach for significantly improving their performance. Our experiments with three LLMs show that this approach consistently improves their simple few-shot performance by roughly 10%, pushing the accuracy of LLMs towards SOTA or surpassing it. On the holdout test set of Spider, the SOTA, in terms of execution accuracy, was 79.9 and the new SOTA at the time of this writing using our approach is 85.3. Our approach with in-context learning beats many heavily fine-tuned models by at least 5%. Additionally, when evaluated on the BIRD benchmark, our approach achieved an execution accuracy of 55.9%, setting a new SOTA on its holdout test set.
URL: https://arxiv.org/abs/2304.11015
Suggested labels
{'label-name': 'Text-to-SQL', 'label-description': 'Focuses on generating SQL queries from natural language text.', 'confidence': 76.74}
The text was updated successfully, but these errors were encountered: