Metadata-Version: 2.4
Name: fast-pysqlparse
Version: 0.7.0
Summary: A fast SQL parser with Python wrapper and C++ core
Author-email: NohaltSial <sail2345@qq.com>
License: MIT
Project-URL: Homepage, https://github.com/Nohaltsail/fast-pysqlparse
Project-URL: Repository, https://github.com/Nohaltsail/fast-pysqlparse.git
Project-URL: Issues, https://github.com/Nohaltsail/fast-pysqlparse/issues
Keywords: sql,parser,ast,fast
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# fast-pysqlparse: High-Performance SQL Parsing Library

[![Build Status](https://img.shields.io/badge/build-passing-brightgreen)]()
[![Language](https://img.shields.io/badge/language-Python%20%7C%20C++17-blue)]()
[![License](https://img.shields.io/badge/license-MIT-purple)]()


[README.md (Chinese)](https://github.com/Nohaltsail/fast-pysqlparse/blob/main/README_CN.md)

A high-performance, cross-platform SQL parsing library, designed to handle the most complex SQL queries with ease.

## Overview

This library provides a robust set of tools for parsing and analyzing SQL statements. Built with a core engine in C++17 for maximum performance, it offers native Python bindings, making it the ideal choice for data-intensive applications where speed and accuracy are critical.

It excels at parsing extremely long SQL statements and queries with deeply nested subqueries, delivering performance far superior to pure-Python alternatives.

The parser is primarily tested against MySQL-style SQL. Current supported dialect names are exposed in `fastsqlparse.conf.SUPPORT_DIALECTS`.

## Features

- **Fast SQL Parsing**: Leverages a high-performance C++17 core to parse SQL statements rapidly
- **Cross-Platform**: Compiled into native extensions (.pyd for Windows, .so for Linux)
- **Comprehensive SQL Support**: Supports a wide range of SQL statements, including:
  - SELECT (with complex JOIN, WHERE, GROUP BY, subqueries, etc.)
  - INSERT
  - Data Definition Language (CREATE)
  - VIEW
  - DELETE
  - UPDATE
  - Common Table Expressions (CTEs), including nested CTEs
- **Abstract Syntax Tree (AST)**: Generates a detailed JSON representation of the parsed SQL AST for easy traversal and analysis
- **SQL Formatting**: Automatically reformats messy SQL into a clean, readable structure
- **Table Lineage Parsing**: Automatically traces and reveals the source-to-target relationships between tables (data lineage)
- **Tokenization**: Breaks down SQL statements into their fundamental tokens for lexical analysis
- **Python API**: A clean and intuitive Python library built around the high-speed native extension

## Performance

This library is engineered for speed. By moving the computationally intensive parsing work to a native C++ layer, it significantly outperforms pure-Python parsing libraries, especially when dealing with large, complex SQL scripts.

### Benchmark Results

#### Test 1: 5000 Iterations
- SQL Length: 600+ characters
- Total Time: 0.60s
- **PPS (Parses Per Second): 8000+**
- Average per parse: 0.1ms~0.15ms

#### Test 2: 10 Million Character SQL
- SQL Length: 10,000,000 characters
- Total Time: 1.40s
- **CPS (Characters Per Second): ~7,500,000**
- Parse successful!

#### Test 3: Python-Only Comparison on ~10M PostgreSQL SQL (No C Benchmark)

Benchmark script: `test/python_parsers_10m.py`

| Parser | Avg Time | CPS |
|--------|----------|-----|
| **fastsqlparse** | 1.3054s | 7,660,928.40 |
| pglast | 4.3322s | 2,308,429.85 |
| sqlglot (postgres) | 22.9163s | 436,392.53 |
| sqlparse | 87.2098s | 114,671.60 |

Notes:
- SQL size: 10,000,484 chars
- Runs per parser: 1
- Results source: results of `test/python_parsers_10m.py`

## Installation

```shell
pip install fast-pysqlparse
```

From Source:
    
    git clone https://github.com/Nohaltsail/fast-pysqlparse.git
    cd fast-pysqlparse
    pip install build
    python -m build
    cd dist
    pip install fast_pysqlparse-*.whl

### Quick Start


```python
from fastsqlparse import Parsed, ParsedQuery

if __name__ == '__main__':
    sql = """

-- main query
SELECT 
    'Monthly Sales Report' AS report_type,
    ms.year,
    ms.month,
    ms.region,
    ms.customer_segment,
    ms.unique_customers,
    ms.total_orders,
    ms.gross_sales,
    ms.avg_order_value,
    ms.cancelled_orders,
    (SELECT SUM(gross_sales) FROM sub_monthly_sales WHERE year = ms.year AND month = ms.month) AS total_monthly_sales,
    ms.gross_sales / NULLIF((SELECT SUM(gross_sales) FROM monthly_sales WHERE year = ms.year AND month = ms.month), 0) * 100 AS sales_percentage,
    (SELECT AVG(avg_order_value) FROM monthly_sales WHERE year = ms.year AND month = ms.month) AS overall_avg_order_value
FROM monthly_sales ms

UNION ALL

SELECT 
    'Category Performance' AS report_type,
    cs.year,
    cs.month,
    NULL AS region,
    cs.category_name AS customer_segment,
    cs.unique_buyers AS unique_customers,
    cs.order_count AS total_orders,
    cs.total_sales AS gross_sales,
    cs.total_sales / NULLIF(cs.order_count, 0) AS avg_order_value,
    NULL AS cancelled_orders,
    (SELECT SUM(total_sales) FROM sub_category_sales WHERE year = cs.year AND month = cs.month) AS total_monthly_sales,
    cs.total_sales / NULLIF((SELECT SUM(total_sales) FROM category_sales WHERE year = cs.year AND month = cs.month), 0) * 100 AS sales_percentage,
    NULL AS overall_avg_order_value
FROM category_sales cs
LIMIT 50, 100"""

    sql_len = len(sql)
    print("sql length: ", sql_len)

    # parse sql statements to SQL object
    sql_stmt = Parsed(sql)
    # Format and print the SQL statement with proper indentation
    print(sql_stmt.format())  # Output formatted SQL statement

    # Tokenization - returns list of tuples containing (token_value, token_type, position)
    tokens = ParsedQuery.tokenize(sql)  # Get tuple list of token information (token_value, token_type, position)

    # Alternative tokenization - returns list of token objects with attributes
    token_obj_list = sql_stmt.tokens()  # Get object list of token information

    # Generate and print Abstract Syntax Tree (AST) in JSON format
    print(sql_stmt.AST())  # Get JSON structure of the SQL statement

    # Extract table lineage/dependencies from the query
    src_tables = ParsedQuery.parse_dependence(sql)  # Get source tables (dependencies) of the query

```

### Comment Handling (`pure`)

`pure` controls SQL comment handling in parser constructors such as `Parsed`,
`ParsedQuery`, `ParsedInsert`, `ParsedCTE`, and `ParsedView`.

- `pure=False` (default): keep comments in parsing/formatting output.
- `pure=True`: strip `--` and `/* ... */` comments before parsing; formatted
  output and token results exclude comments, and parsing may be faster.

```python
from fastsqlparse import Parsed

parsed_keep = Parsed(sql, pure=False)  # preserve comments
parsed_clean = Parsed(sql, pure=True)  # strip comments before parse
```

## When to Use Which Parser

| **Scenario** | **Parser to Use**  |
|--------------|--------------------|
| SQL statement type is unknown or you don't want to specify the type | `Parsed/ParsedOne` |
| Multiple SQL statements separated by `;` (script execution) | `Parsed`           |
| SELECT / query statement | `ParsedQuery`      |
| INSERT statement | `ParsedInsert`     |
| DELETE statement | `ParsedDelete`     |
| UPDATE statement | `ParsedUpdate`     |
| CREATE TABLE statement | `ParsedCreate`     |
| CREATE VIEW statement | `ParsedView`       |
| CTE (WITH clause) statement | `ParsedCTE`        |

> **Note:** If your SQL contains multiple statements separated by semicolons (e.g., a script with CREATE, INSERT, SELECT), you **must** use `Parsed`. The type-specific parsers are designed for single, known-type statements only.

## Documentation

For complete API documentation, see: [API_DOC.md](https://github.com/Nohaltsail/fast-pysqlparse/blob/main/API_DOC.md)

## Contributing

Contributions are welcome! Please feel free to submit pull requests, report bugs, or suggest new features.

## License

This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details.

### Note on Dynamic Libraries

This project currently distributes precompiled dynamic libraries (`.pyd` and `.so`).
The corresponding C++ source code for these dynamic libraries is temporarily not public and is planned to be opened in a future release.

For the full supplementary notice, see [LICENSE](LICENSE).

