Your SQL parser doesn't know your schema

When you add a validation layer to LLM-generated SQL, the obvious choice is a parser library. They are well maintained, fast, and give you a clean parse tree. However, they cannot catch failures that actually matter, because they do not know your schema.

What we were building

Conversational agents built on top of production databases need to answer open-ended questions: how many roles did we close last quarter, show candidates we interviewed last month, what is the average time to fill for engineering positions. The question space is too large to enumerate as fixed queries, which is why the standard approach is to have the model generate SQL on demand. The user asks in natural language, the model writes a query against the schema, the query runs, and the results come back as part of a conversational response.

This works. It also introduces a problem: the SQL that runs against your production database was written by a model, not by you. You need to verify the query before execution, and not just that it is safe. You need to know it can actually run.

The other safety layers, a SELECT-only classifier, a deterministic user-scoping check, and a table allowlist, handle scope and access, and are covered in a companion post. What none of them handles is whether the query the model generated can execute against the schema as it exists today. That is the validation question this post is about.

What we considered

A dedicated SQL parser library validates grammar and syntax. It is fast, offline, and needs no database connection. It can only confirm whether the SQL is syntactically valid.

EXPLAIN uses the database's own query planner, run against the query without executing it. It fails on syntactically correct SQL the database cannot execute because of nonexistent columns, type mismatches, unresolvable identifiers.

We chose EXPLAIN.

Why we chose EXPLAIN

The failure modes that matter for LLM-generated SQL are not grammar errors. The model knows SQL syntax. What it does not know is your current schema. It generates queries that are syntactically correct and semantically broken: a column referenced by its old name, a join on a field it inferred from context, a filter using the wrong type comparison.

Take something simple. A query selecting a column named email, when that column was renamed to email_address three migrations ago.

-- Grammatically valid. Any parser library accepts this.
SELECT email FROM candidates WHERE id = 42;

-- Postgres, via EXPLAIN, does not:
ERROR:  column "email" does not exist
LINE 1: SELECT email FROM candidates WHERE id = 42;
               ^
HINT:  Perhaps you meant to reference the column "candidates.email_address".

EXPLAIN catches this because it runs the actual query planner against the actual schema. When a column does not exist, EXPLAIN says so. "Can Postgres plan this against the current schema?" is a different question from "Is this valid SQL syntax?", and it is the right question when the SQL was generated by a model.

There is a second reason, separate from schema awareness. Any SQL parser library is an approximation of Postgres's own parser, and the gap is where edge cases hide: quoted identifiers, recent syntax additions, dialect-specific behaviour. EXPLAIN uses the same parser that will execute the query. There is no gap by definition. Whatever Postgres accepts, EXPLAIN accepts. Whatever Postgres rejects, EXPLAIN rejects.

The practical cost is negligible: one round trip in a flow already dominated by seconds of model generation. We run EXPLAIN, catch errors, retry generation once if needed, and only proceed to execution after EXPLAIN succeeds.

Use plain EXPLAIN, never EXPLAIN ANALYZE

This deserves its own section because the instinct runs the wrong way. Most engineers meet EXPLAIN through performance work, where ANALYZE is the default habit, and ANALYZE executes the statement.

-- Plans the query. Does not run it. This is the validation step.
EXPLAIN SELECT count(*) FROM roles WHERE closed_at > now() - interval '90 days';

-- Runs the statement, then reports what happened.
-- Never use this as a validation step.
EXPLAIN ANALYZE DELETE FROM candidates WHERE id = 42;

On a SELECT, ANALYZE costs you a wasted query. On anything else, it is the failure your validation layer existed to prevent, executed by the validation layer itself. The classifier upstream should never let a DELETE reach this point, but a safety layer that depends on another safety layer never failing is not a safety layer. Assert plain EXPLAIN in code, not in a comment.

Stronger still: give the connection a role with SELECT-only grants. A database privilege is deterministic where a classifier is probabilistic, and it makes this whole class of mistake unreachable.

What EXPLAIN does not catch

Plannable is not the same as executable. EXPLAIN validates the plan, not the run. Division by zero, a cast that fails on actual data, integer overflow, a statement timeout: all of these plan cleanly and fail at execution. Keep the error handling on the execution path; EXPLAIN narrows the failure surface, it does not close it.

It also cannot tell you the query answers the question that was asked. Filtering on created_at when the user meant closed_at plans fine, runs fine, and returns a confidently wrong number into a conversational response. That is an evaluation problem, not a validation problem, and it needs a different mechanism.

And it says nothing about what comes back. Query results flow into a model-generated response, which means the content of your database rows reaches the model. A query can be perfectly valid and return text that was written to influence whatever reads it. Validating the query is not validating the data.

How it held up

Well, and unexpectedly so. The validation layer does not just catch bad queries; it produces an observable signal. Queries that fail EXPLAIN are logged separately from queries that succeed, which means schema drift is visible. When a column is renamed in a migration, the failure rate on that query pattern becomes detectable before any end user sees an error. A library parser would have silently accepted those queries throughout.

That observability turned out to be as valuable as the validation itself. It changes schema drift from a silent runtime problem into something you can see at the query generation layer.

Where we would extend it

One thing we are currently implementing; We pay for the round trip and keep only pass or fail. EXPLAIN also returns the planner's cost estimate and expected row count, and we throw both away.

EXPLAIN (FORMAT JSON)
SELECT * FROM candidates c JOIN applications a ON a.candidate_id = c.id;

-- "Node Type": "Seq Scan"
-- "Total Cost": 184203.55
-- "Plan Rows": 2841003

A query can plan perfectly and still be a sequential scan across your largest table. That risk is structural here, because the question space is unbounded by design: nobody wrote the query, so nobody sized it. Reading Total Cost from the plan you already fetched and rejecting or flagging above a threshold costs no extra round trip and closes a hole a parser could never have closed either. Pair it with a statement timeout on the execution path as the hard backstop.

What we learned

Pick the check whose definition of correct matches the property you actually need verified. "Syntactically valid SQL" and "SQL Postgres can plan against this schema" sound like the same thing. They are not. Use the check that answers your actual question, be precise about what it does not answer, and if it also hands you production observability for free, that is a better return than fast offline validation.