Metadata-Version: 2.4
Name: trino-sql-validator
Version: 0.15.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Rust
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: pytest>=7 ; extra == 'dev'
Requires-Dist: ruff>=0.1 ; extra == 'dev'
Requires-Dist: mypy>=1 ; extra == 'dev'
Provides-Extra: dev
License-File: LICENSE
Summary: Fast Trino SQL syntax validator — Python library backed by Rust
Keywords: trino,sql,validator,pyo3,rust
Author: Ivan
License: MIT
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/ivanshamaev/trino-sql-validator/blob/main/CHANGELOG.md
Project-URL: Homepage, https://github.com/ivanshamaev/trino-sql-validator
Project-URL: Issues, https://github.com/ivanshamaev/trino-sql-validator/issues
Project-URL: Repository, https://github.com/ivanshamaev/trino-sql-validator

# trino-sql-validator

Fast **Trino SQL syntax validator** — a Python library whose core is written in
Rust and compiled to a native extension via [PyO3] + [maturin].

Installable from PyPI:

```bash
pip install trino-sql-validator
```

## Quickstart

```python
from trino_sql_validator import analyze_statements, validate, validate_file

# A string with one or many statements
result = validate("SELECT 1; SELECT * FROM t WHERE a > 0;")
assert result.valid
assert result.statement_count == 2

# Invalid SQL returns a value, never raises
result = validate("SELECT * FORM t")
assert not result.valid
print(result.error)          # e.g. "Expected: end of statement, found: FORM at line 1, column 10"
print(result.error.line)     # 1

# Validate a file
result = validate_file("queries.sql", dialect="trino")

# Opt-in per-statement metadata; indexes are zero-based
analysis = analyze_statements("EXPLAIN SELECT 1; CALL system.custom_proc()")
assert analysis.validation.valid
assert analysis.statements[0].kind == "explain"
assert analysis.statements[0].inner_kind == "query"
assert analysis.statements[1].kind == "call"
```

Invalid SQL (and files containing it) is returned as a `ValidationResult`;
it is **not** raised as an exception. Only genuine misuse (unknown dialect,
unreadable file) raises.

### Catalog warnings (functions and data types)

For `dialect="trino"`, `validate()` also checks that every function called and
every data type used in the SQL exists in the documented Trino catalog. Unknown
names are reported as non-fatal `warnings` — `valid` stays `True` because syntax
is fine:

```python
result = validate("SELECT marh(1.5)")       # round() misspelled
assert result.valid
print(result.warnings)                      # (FunctionWarning(name='marh', line=1, column=8),)
print(result.unknown_functions)             # ['marh']

result = validate("CREATE TABLE t (a bignum, b bigint)")  # bigint vs bignum
print(result.warnings[0])                   # TypeWarning(name='bignum', line=1, column=19)
print(result.unknown_types)                 # ['bignum']
```

The catalogs are auto-generated from the Trino docs and only check *name
existence*, not argument counts, precision/scale, or semantic correctness.
`hive`/`generic` dialects skip these checks. False positives are possible if a
deployed Trino adds plugin functions/types beyond the docs.

Inline `WITH FUNCTION` names are exempt only within their own query scope.
Qualified calls with the same final name are still checked. Procedure names in
`CALL` and `ALTER TABLE ... EXECUTE` are not scalar functions and therefore do
not produce `FunctionWarning`; their existence, parameters, arity, permissions,
and connector availability require a Trino coordinator and are out of scope.

### Statement metadata

`analyze_statements()` is an opt-in API that returns the unchanged
`ValidationResult` together with a tuple of `StatementInfo`. Each entry contains
a zero-based index, source span, source-derived kind, and (for `EXPLAIN` or
`PREPARE`) an `inner_kind` when it can be identified. On invalid multi-statement
input, `error_statement_index` identifies the source statement when the parser
provided a location. Existing `validate()` and `validate_file()` return types
are unchanged.

### dbt and Jinja templates

Jinja/dbt SQL is supported by default. `validate()` and `validate_file()` use
`jinja="auto"` to mask Jinja expressions, statements, and comments before
parsing while preserving line numbers and file structure. This supports
constructs such as `{{ ref("orders") }}` and `{{ var("catalog") }}` without
requiring a dbt installation or project context. Use `jinja="mask"` as an
explicit spelling of the same mode.

Use `jinja="reject"` to pass the original template directly to the SQL parser.
Masking cannot determine SQL generated by control-flow blocks, macros, or
adapter semantics; render those cases with dbt and validate the rendered SQL
for complete coverage.

### Dialects

- `"trino"` (default) — Trino-flavored with a custom override tuned for
  current Trino syntax, including Iceberg branches/time travel, complex nested
  types, routines, table functions, SQL/JSON, and Trino-specific DDL.
- `"hive"` and `"generic"` — offered as permissive alternates.

## Known limitations

`sqlparser-rs` (the parser we use) performs **syntax** validation, not semantic
analysis. It may accept SQL that Trino would reject at analysis time (unknown
columns/tables, duplicate columns), and it can reject exotic Trino-specific DDL.
The validator has targeted compatibility parsing for documented Trino syntax,
including nested `ROW`/`ARRAY`/`MAP` types, but it does not replace Trino's
semantic analyzer. See [`plan/roadmap.md`](plan/roadmap.md) for the path toward
stricter Trino fidelity.

Parser fidelity is checked reproducibly against direct-string cases extracted
from Apache Trino's parser tests. With ordinary Java strings and text blocks,
the pinned Trino 483 audit currently accepts 476/484 statements, 231/238
expressions, 68/68 types, the extracted Functions/Routines subset, and rejects
23/23 direct negative statements. Known differences are pinned in a named
allowlist; new mismatches or a reduced extracted denominator fail the audit.
These figures and the 276 independently checked positive fixture statements
describe measured corpora, not complete Trino grammar or connector behavior.
SQL embedded inside ordinary string literals, JSON paths, WKT, dynamic SQL, and
unrendered macro output is intentionally opaque rather than recursively parsed.

To keep invalid or adversarial input from exhausting the native parser stack,
validation rejects a statement after 4,096 significant SQL tokens, nesting
deeper than 256 groups or routine blocks, and an input after 65,536 significant
tokens. The failure is returned as an ordinary invalid `ValidationResult`; when
the limiting token has a source position, that position is included in the
error. Semicolon-separated statements have independent per-statement budgets.

## Development

See [`AGENTS.md`](AGENTS.md) for setup, internal conventions, and release steps.
Key commands:

```bash
python3 -m venv .venv && source .venv/bin/activate
pip install -U pip maturin && pip install -e ".[dev]"
maturin develop          # build + install native ext into the venv
cargo test               # Rust tests
pytest -q                # Python tests
cargo fmt --check        # formatting
cargo clippy --all-targets -- -D warnings
python tools/extract_functions.py --ref 483 --check
python tools/extract_types.py --ref 483 --check
python tools/audit_upstream_parsers.py --baseline plan/trino_483_audit_baseline.json --fail-on-regression
```

## License

MIT

[PyO3]: https://pyo3.rs
[maturin]: https://maturin.rs

