Model Tag:

DuckDB-NSQL: Transforming Natural Language into SQL Queries

DuckDB-NSQL is a sophisticated text-to-SQL model comprising 7 billion parameters, meticulously created by MotherDuck and Numbers Station. This model is designed with the specific purpose of generating SQL queries from natural language prompts, making it an invaluable tool for developers and data analysts alike.

Overview of DuckDB-NSQL

DuckDB-NSQL is built upon Meta’s original Llama-2 model, which is a powerful transformer architecture. It has undergone additional pre-training on a comprehensive dataset of general SQL queries and has been fine-tuned on a specialized dataset containing text-to-SQL pairs uniquely relevant to DuckDB.

What Makes DuckDB-NSQL Unique?

This model stands out due to the following features:

• Versatile SQL Generation: Unlike many existing text-to-SQL models that are limited to generating SELECT statements, DuckDB-NSQL can produce any valid SQL command compatible with DuckDB, including data manipulation and definition statements.
• Extensive Training Data: The model benefits from over 200,000 synthetically generated text-to-SQL pairs that leverage the robust features of DuckDB.
• Enhanced Performance: The training process optimizes the model's ability to comprehend complex prompts and respond accurately.

Usage of DuckDB-NSQL

To utilize DuckDB-NSQL for SQL generation, users must provide a schema and specific prompts. Here’s a basic example:

Example Prompt

    Provided this schema:
    CREATE TABLE orders (
        OrderID bigint,
        CustomerID bigint,
        OrderDate timestamp,
        ProductName varchar(255),
        Quantity int,
        PricePerUnit double,
        TotalAmount double
    );
    Give me orders placed after January 1, 2023
    

Example Output

    SELECT * FROM orders WHERE OrderDate > '2023-01-01';
    

Training Data and Process

The model has been fine-tuned on a dataset consisting of:

• 200,000 DuckDB text-to-SQL pairs, generated using Mixtral-8x7B-Instruct-v0.1 and guided by DuckDB v0.9.2 documentation.
• Text-to-SQL pairs transpiled to DuckDB SQL using sqlglot.

During the training, cross-entropy loss was utilized to maximize the sequential input likelihood while minimizing overfitting on the SQL components of the pairs.

Intended Use

DuckDB-NSQL is primarily designed for generating SQL queries from specified table schemas and natural language instructions. It excels when used with the defined prompt formats and is versatile in producing a range of SQL statements beyond simple SELECT queries.

For those looking to leverage local language models for database querying, DuckDB-NSQL seamlessly integrates into various workflows.