SQLMorph is a research framework for evaluating Text-to-SQL systems with greater depth and precision. It introduces two query mutation techniques — Join Query Expansion (JQE) and Textual Query Augmentation (TQA) — to automatically expand evaluation sets and expose model weaknesses. It also defines a family of fine-grained execution metrics (Execution Precision, Execution Recall, and F1) that reveal over- and under-prediction patterns hidden by binary measures like Execution Accuracy.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/dais-polymtl/sqlmorph/llms.txt
Use this file to discover all available pages before exploring further.
Introduction
Learn what SQLMorph does, how it works, and when to use it.
Quickstart
Install SQLMorph and run your first evaluation in minutes.
Core Concepts
Understand JQE, TQA, and the fine-grained metrics that power SQLMorph.
API Reference
Explore the Python API for the Evaluation class and query mutation functions.
What SQLMorph provides
Join Query Expansion
Automatically expand SQL queries with semantically valid joins to probe structural complexity limits.
Textual Query Augmentation
Generate controlled natural language perturbations to measure robustness to linguistic variation.
Fine-Grained Metrics
Move beyond binary EX with Execution Precision, Execution Recall, and F1 scoring.
Multi-LLM Support
Connect to OpenAI, HuggingFace, Ollama, Google Gemini, and Together AI backends.
Get started
Configure your environment
Copy
.env.example to .env and add your OpenAI API key for semantic metrics and NL generation.Run an evaluation
Use the
Evaluation class or CLI to compare a predicted SQL query against a ground truth and get precision, recall, and F1 scores.