The landscape of natural language processing has long sought a seamless bridge between human intent and structured database queries. While the industry has historically approached the "Text-to-SQL" challenge as a purely linguistic translation task, Feyn AI, a Y Combinator-backed startup, has unveiled a specialized model family that shifts this paradigm toward active inspection. Known as SQRL, this new suite of models is designed to interrogate a database before committing to a final query, a method that addresses the chronic inaccuracies found in traditional translation-based systems. By prioritizing database inspection over immediate generation, the Feyn AI team reports that their flagship model, SQRL-35B-A3B, has achieved a 70.6% execution accuracy on the rigorous BIRD Dev benchmark, effectively surpassing the performance of frontier models like Claude 3.5 Opus.
The Evolution of Text-to-SQL from Translation to Investigation
For years, the development of Text-to-SQL systems followed a predictable trajectory. Early models functioned much like language translators, attempting to map the syntax of natural language directly onto the rigid structure of SQL. However, this approach frequently faltered when faced with real-world database complexities. A query may be syntactically perfect—meaning it follows all the rules of SQL and will execute without error—yet still be logically incorrect. The fundamental issue lies in the disconnect between a database’s schema (its structural blueprint) and the actual data values contained within its rows.
Standard schemas provide the names of tables and columns but rarely offer insight into the specific formatting of the data. For instance, a "Location" column might store a county as "Alameda," "Alameda County," or "ALAMEDA." Without knowing which convention is used, a model is forced to guess, often resulting in "silent failures" where a query runs but returns zero results or, worse, incorrect data. Furthermore, complex joins between tables can lead to duplicate rows or incorrect aggregations if the model does not understand the underlying data relationships. Feyn AI’s SQRL addresses this by treating the database not just as a destination for a query, but as a source of truth that can be consulted during the reasoning process.
Strategic Methodology: The Inspection Loop
The SQRL model family introduces a "look-before-you-leap" architecture. When presented with a natural language question, the model evaluates whether the provided schema and context are sufficient to generate an accurate answer. If the prompt remains ambiguous, the model initiates a series of read-only exploratory queries. These queries are designed to probe the database for specific values, row counts, or data distributions that clarify the user’s intent.
The interaction is structured through a specialized protocol involving distinct tags. The model uses <sql> blocks to request preliminary observations from the database. A separate execution harness processes these requests in a secure, read-only environment and returns the results within <observation> tags. Once the model has gathered sufficient information, it utilizes an <answer> block to finalize the SQL query. This iterative process can occur up to five times, though Feyn AI notes that most queries are resolved in significantly fewer steps. This "circumstantial inspection" ensures that simple questions—such as counting rows in a single, well-defined table—remain computationally inexpensive, while complex or ambiguous requests receive the necessary investigative depth.
Training Architecture and Dataset Refinement
The development of SQRL required a departure from standard fine-tuning practices. Teaching a model when to inspect a database, rather than just how to write SQL, necessitates a sophisticated training environment. The Feyn AI team began by auditing the industry-standard BIRD and Spider datasets. Recognizing that even these benchmarks contain flawed reference queries, the team utilized a panel of three model judges to prune the training pool. They removed any examples where the reference SQL failed to produce a usable result or did not accurately answer the accompanying question.
To instill the decision-making logic required for inspection, the SQRL-35B-A3B teacher model was trained using Constrained Importance Sampling Policy Optimization (CISPO), a reinforcement learning technique derived from MiniMax’s M1 research. Unlike traditional methods that clip the policy ratio, CISPO clips importance-sampling weights, which allows the model to retain critical gradient signals from rare but decisive tokens. During training, the model generated multiple trajectories for each question. Feyn AI rewarded trajectories that resulted in a correct data return, regardless of the specific SQL wording, creating a binary success signal based on execution rather than stylistic matching.
A critical component of this training was the "mixed zone" strategy. By focusing on scenarios where some attempts succeeded and others failed, the reinforcement learning process could isolate the specific investigative steps that led to a correct answer. This high-quality teacher data was then distilled into smaller "student" models—SQRL-9B and SQRL-4B—which were fine-tuned on the teacher’s reasoning paths and further refined through CISPO execution rewards.
Benchmarking Success Against Industry Giants
The efficacy of the SQRL approach is most evident in its performance on the BIRD (Big Bench for Large-scale Database Grounded Text-to-SQL Evaluation) benchmark. BIRD is widely considered the most challenging test for Text-to-SQL systems because it utilizes large, real-world databases with messy data and complex relational structures.
In the BIRD Dev evaluation, SQRL-35B-A3B achieved an execution accuracy of 70.60%. This performance is particularly notable when compared to general-purpose frontier models. Under the same evaluation parameters, Claude 3.5 Opus reached 68.77%, while Claude 3.5 Sonnet scored 67.34%. Even significantly larger models, such as the Qwen2.5-Coder-480B-A35B, trailed behind at 66.17%.
The efficiency of the SQRL family is also a key differentiator. The SQRL-9B model retained nearly the same level of accuracy as its larger counterpart, scoring 69.80%. Perhaps most impressively, the SQRL-4B model achieved 68.80% accuracy—matching the performance of Claude 3.5 Opus while being small enough to be hosted on modest hardware. This democratization of high-performance Text-to-SQL allows organizations to maintain data privacy by running these models on their own local infrastructure rather than relying on third-party API providers.
Deployment and Practical Implementation
Feyn AI has made the SQRL family openly available via Hugging Face, providing checkpoints for the 4B, 9B, and 35B versions. The models are built upon the Qwen architecture and are compatible with standard serving frameworks like vLLM. For enterprise deployment, Feyn AI recommends the 9B model as the optimal balance between speed and accuracy, while the 4B model is suited for environments with limited GPU resources.
Implementation of SQRL requires a specialized application loop. Developers must ensure that the database connection provided to the model is strictly read-only to prevent accidental data modification during the inspection phase. Furthermore, because the model’s reasoning and action protocol occur after the standard <think> tags used in reasoning models, serving layers must be configured not to strip content following those tags. Feyn AI has provided a reference harness and system prompts to assist developers in integrating the inspection loop into their existing data stacks.
Broader Implications for the AI and Data Industry
The release of SQRL marks a significant shift in how the AI industry views the intersection of natural language and structured data. By moving away from a "black box" generation model and toward a transparent, investigative agent, Feyn AI is addressing the trust gap that often prevents enterprises from fully adopting AI-driven data analysis.
The ability of a 4B parameter model to match the performance of a frontier model like Claude 3.5 Opus suggests that task-specific optimization and innovative reasoning loops can outperform raw scale. This has profound implications for the cost of AI operations. Organizations can now deploy highly accurate SQL agents at a fraction of the cost previously required for frontier model API calls.
Moreover, the "inspection-first" philosophy aligns with broader trends in AI safety and reliability. By requiring the model to verify its assumptions against the actual database, the likelihood of "hallucinated" columns or tables is drastically reduced. As businesses increasingly rely on data-driven decision-making, the demand for systems that can navigate the nuances of real-world databases with high precision will only continue to grow. SQRL represents a foundational step toward a future where natural language becomes the primary interface for data exploration, without sacrificing the rigor and accuracy required by professional data analysts.
