Metadata-Version: 2.4
Name: scan-google-sheet
Version: 0.3.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Typing :: Typed
Requires-Dist: polars>=1.0
License-File: LICENSE
Summary: Read public Google Sheets into Polars DataFrames and LazyFrames — no auth required, powered by Rust
Keywords: polars,google-sheets,csv,dataframe,rust
Author-email: Attica-oss <g.mounac@gmail.com>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Issues, https://github.com/Attica-oss/scan_google_sheet/issues
Project-URL: Repository, https://github.com/Attica-oss/scan_google_sheet

# Scan Google Sheet

Read public Google Sheets into Polars DataFrames and LazyFrames — no auth, no service accounts, no API keys.

[![PyPI](https://img.shields.io/pypi/v/scan-google-sheet)](https://pypi.org/project/scan-google-sheet/)
[![Python](https://img.shields.io/pypi/pyversions/scan-google-sheet)](https://pypi.org/project/scan-google-sheet/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

---

## 🦀 Now powered by Rust

This is a from-scratch rewrite of the original pure-Python `scan-google-sheet`. The
public API is unchanged — `read_google_sheet`, `scan_google_sheet`, and the same
exception hierarchy all work exactly as before — but the fetch-and-parse core
underneath is now Rust, compiled to a native extension via [PyO3](https://pyo3.rs)
and [maturin](https://www.maturin.rs/).

| | Before (0.1.x) | Now (0.2.0) |
|---|---|---|
| HTTP client | `httpx` (Python) | `reqwest` (Rust, blocking + rustls) |
| CSV parsing | `polars` (Python API) | `polars` (Rust, native) |
| GIL behaviour | Held during fetch + parse | Released for the whole fetch + parse |
| Distribution | Pure-Python wheel | Compiled extension module (`_core`) per platform |
| Public API | `read_google_sheet`, `scan_google_sheet` | Unchanged — drop-in upgrade |
| Errors | Python exception hierarchy | Same hierarchy, raised from Rust |

**What didn't change:** if you're upgrading from `0.1.x`, this is a drop-in
replacement — same function signatures, same return types, same exceptions.
Nothing in your calling code needs to change.

**On performance — set expectations correctly:** for a single call, don't
expect a dramatic speedup. Fetching a public sheet is dominated by TLS
handshake time and Google's own server-side render of the sheet to CSV —
typically 90%+ of total latency — neither of which any client-side language
touches. The original package's CSV parsing was also already Rust under the
hood (`polars`'s Python bindings call into the same Rust crate this rewrite
calls directly), so there was never a slow "Python parser" to replace. What
this rewrite actually changes:

- The Python GIL is released for the whole fetch + parse (`Python::detach`),
  so multiple threads making concurrent `read_google_sheet` calls now run
  those requests in parallel instead of serialising on the GIL.
- HTTP connections are pooled in a single process-wide `reqwest::Client`
  (see `src/core/fetch.rs`), so repeated calls in one process reuse
  keep-alive connections instead of re-negotiating TLS every time — the one
  lever that meaningfully cuts latency for an app calling this repeatedly.
- One fewer FFI hop per call (no intermediate Python `str` handed to
  `pl.read_csv`), which shaves microseconds, not milliseconds.

If you're benchmarking, measure many concurrent/repeated calls in one
process, not a single cold fetch — a single fetch's wall time is mostly
Google's server, not this library.

---

## Requirements

- Python ≥ 3.9
- The spreadsheet must be set to **Anyone with the link can view**

---

## Installation

```bash
pip install scan-google-sheet
# or
uv add scan-google-sheet
```

Prebuilt wheels are published for common platforms/Python versions; pip falls
back to building from source (requires a Rust toolchain) if no matching wheel
is available.

---

## Quick start

```python
from scan_google_sheet import read_google_sheet, scan_google_sheet
```

**Eager — returns a `DataFrame` immediately:**

```python
df = read_google_sheet("Sheet1", sheet_id="1BxiMVs0XRA5nFMdKvBdBZjgm...")
```

**Lazy — returns a `LazyFrame`, participates in Polars query optimisation:**

```python
lf = scan_google_sheet("Sheet1", sheet_id="1BxiMVs0XRA5nFMdKvBdBZjgm...")

df = (
    lf
    .filter(pl.col("year") == 2025)
    .select("vessel", "amount")
    .collect()
)
```

You can also pass a full Google Sheets URL instead of a bare sheet ID:

```python
df = read_google_sheet(
    "Sheet1",
    url="https://docs.google.com/spreadsheets/d/1BxiMVs0.../edit#gid=0",
)
```

**Server-side query — only fetch the rows/columns you need:**

```python
# Columns are referenced by spreadsheet letter (A, B, C...), not header name
df = read_google_sheet(
    "RawData",
    sheet_id="1BxiMVs0XRA5nFMdKvBdBZjgm...",
    query="select A, C, G, H where YEAR(K) = 2026",
)
```

`query` is a [Google Visualization API Query Language](https://developers.google.com/chart/interactive/docs/querylanguage)
string, applied by Google before the data is downloaded — smaller transfer,
less to parse.

**Detecting an active sheet filter:**

Google's export endpoint (`/export?format=csv`) always returns every row,
but the endpoint `read_google_sheet` uses to support `sheet_name`/`query`
(`/gviz/tq`) silently *excludes* rows hidden by an active filter on that tab.
If someone left a filter on, your query can quietly return less data than
you expect. Pass `warn_if_filtered=True` to check for this — it costs two
extra HTTP requests and emits a `UserWarning` if rows appear to be hidden:

```python
import warnings

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    df = read_google_sheet(
        "RawData", sheet_id="1BxiMVs0...", warn_if_filtered=True
    )
    if caught:
        print(caught[0].message)
        # sheet 'RawData' appears to have an active filter hiding 12 row(s)
        # (88 visible vs 100 in the full export) — query results may be incomplete
```

This is a best-effort heuristic, not a guarantee — it's most reliable when
`url` (with `#gid=...`) is passed rather than a bare `sheet_id`, since without
a `gid` the check falls back to comparing against the spreadsheet's first tab.

---

## API

### `read_google_sheet`

```python
def read_google_sheet(
    sheet_name: str,
    sheet_id: str | None = None,
    url: str | None = None,
    *,
    timeout: int = 10,
    parse_dates: bool = True,
    query: str | None = None,
    warn_if_filtered: bool = False,
) -> pl.DataFrame
```

Fetches the sheet and returns a collected `DataFrame`. The HTTP fetch and CSV
parse both run in Rust, released from the Python GIL.

### `scan_google_sheet`

```python
def scan_google_sheet(
    sheet_name: str,
    sheet_id: str | None = None,
    url: str | None = None,
    *,
    timeout: int = 10,
    parse_dates: bool = True,
    batch_size: int = 1_000,
    query: str | None = None,
    warn_if_filtered: bool = False,
) -> pl.LazyFrame
```

Returns a `LazyFrame` registered via the Polars IO plugin API. Projection
pushdown, predicate pushdown, `head()`, and streaming are all supported.

> **Note:** Google Sheets does not support partial HTTP reads. The full sheet
> is always downloaded in one request by the Rust core. Pushdowns reduce
> processing cost, not network cost. This is the one part of the API that
> stays in Python: `register_io_source` is a Polars Python-only hook with no
> Rust-side equivalent, so `scan_google_sheet` is a thin Python wrapper around
> the Rust-backed `read_google_sheet`.

**Parameters shared by both functions:**

| Parameter | Type | Default | Description |
|---|---|---|---|
| `sheet_name` | `str` | — | Tab name as shown in Google Sheets |
| `sheet_id` | `str \| None` | `None` | Spreadsheet ID from the URL |
| `url` | `str \| None` | `None` | Full Google Sheets URL (ID extracted automatically) |
| `timeout` | `int` | `10` | HTTP timeout in seconds |
| `parse_dates` | `bool` | `True` | Attempt automatic date/datetime parsing |
| `query` | `str \| None` | `None` | Google Visualization API Query Language string, applied server-side |
| `warn_if_filtered` | `bool` | `False` | Emit a `UserWarning` if the tab appears to have rows hidden by an active filter (2 extra requests) |

Provide either `sheet_id` or `url`, not both.

---

## URL utilities

```python
from scan_google_sheet import extract_sheet_id, build_gviz_url, from_url

# Extract the sheet ID from any Google Sheets URL
sheet_id = extract_sheet_id("https://docs.google.com/spreadsheets/d/ABC123/edit")
# "ABC123"

# Build a gviz CSV export URL from a sheet ID and tab name
url = build_gviz_url("ABC123", "Sheet1")
# "https://docs.google.com/spreadsheets/d/ABC123/gviz/tq?tqx=out:csv&sheet=Sheet1"

# ...optionally with a query, applied server-side
url = build_gviz_url("ABC123", "Sheet1", "select A, C where B = 1")
# "https://docs.google.com/spreadsheets/d/ABC123/gviz/tq?tqx=out:csv&sheet=Sheet1&tq=select%20A%2C%20C%20where%20B%20%3D%201"

# Build a gviz URL directly from a full Google Sheets URL
url = from_url("https://docs.google.com/spreadsheets/d/ABC123/edit", "Sheet1")
```

---

## Error handling

All exceptions inherit from `ReadSheetError`, so you can catch everything with
one handler or branch on specific types:

```python
from scan_google_sheet import (
    read_google_sheet,
    ReadSheetError,
    SheetFetchError,
    SheetURLError,
    SheetParseError,
    NetworkError,
    ConfigurationError,
)

try:
    df = read_google_sheet("Sheet1", sheet_id="...")
except ReadSheetError as e:
    match e:
        case SheetFetchError() if e.is_auth_error:
            print("Make the sheet public (Share → Anyone with the link)")
        case SheetFetchError() if e.is_not_found:
            print(f"Sheet not found — check the ID: {e.url}")
        case NetworkError():
            print(f"No connection: {e.cause}")
        case SheetURLError(raw=r):
            print(f"Could not parse URL: {r!r}")
        case SheetParseError():
            print(f"CSV parse failed: {e.cause}")
        case ConfigurationError():
            print(str(e))
```

### Exception hierarchy

```
ReadSheetError
├── SheetURLError       malformed URL or unextractable sheet ID  (.raw)
├── SheetFetchError     non-200 HTTP response                    (.url, .status_code)
│                                                                (.is_auth_error, .is_not_found)
├── SheetParseError     CSV or Polars parsing failure             (.column)
├── NetworkError        transport failure, no response received  (.url)
└── ConfigurationError  invalid argument combination
```

Raised from Rust via [PyO3](https://pyo3.rs), but the hierarchy, messages, and
attributes are the same ones the pure-Python `0.1.x` release raised.

---

## Making your sheet public

In Google Sheets: **Share → Change to Anyone with the link → Viewer → Done.**

The export URL used by this library (`gviz/tq?tqx=out:csv`) requires the sheet
to be publicly readable. No data is ever written.

---

## How it works

```
Google Sheets URL / ID
        │
        ▼
  build_gviz_url()          constructs the CSV export URL          [Rust]
        │
        ▼
    fetch_raw()              reqwest GET, pooled client, GIL released [Rust]
        │
        ▼
    parse_csv()              polars CSV → DataFrame                 [Rust]
        │
        ▼
  read_google_sheet()        PyO3 boundary → pl.DataFrame     [Rust → Python]
        │
        ▼
  scan_google_sheet()        register_io_source() lazy wrapper     [Python]
        │
        ▼
  LazyFrame / DataFrame      ready for your pipeline
```

The crate also builds as a standalone Rust library (`cargo build`, no Python
required) — see `src/lib.rs` for the `read_public_sheet` entry point used by
the `python` feature's PyO3 bindings.

---

## Development

```bash
git clone https://github.com/Attica-oss/scan_google_sheet
cd scan_google_sheet

# Rust-only build/check + unit/integration tests (mocked HTTP, no network)
cargo build
cargo test

# Python extension, editable install into a venv
uv venv
source .venv/bin/activate
uv sync --group dev
maturin develop --release --features python

# Python test suite
uv run pytest
```

> `maturin develop`/`maturin build` without `--release` produce an
> unoptimized debug build (no inlining, no bounds-check elision). It won't
> be noticeable on small sheets, but always benchmark and publish `--release`
> builds.

---

## Changelog

### 0.2.0
- Full rewrite of the fetch/parse core in Rust (PyO3 + maturin), GIL released
  during network and parse work via `Python::detach`
- Process-wide pooled `reqwest::Client` (`src/core/fetch.rs`) so repeated
  calls reuse keep-alive connections instead of a fresh TLS handshake each time
- Public API and exception hierarchy unchanged from `0.1.x`, including the
  structured attributes on each exception (`raw`, `url`, `status_code`,
  `is_auth_error`, `is_not_found`, `column`, `cause`)
- `scan_google_sheet`'s lazy IO-plugin wrapper stays in Python (Polars
  `register_io_source` has no Rust-side equivalent)
- Rust unit/integration tests (`cargo test`, mocked HTTP via `mockito`) and a
  Python test suite (`uv run pytest`)

### 0.1.1 and earlier
- Pure-Python implementation (`httpx` + `polars`)
- `read_google_sheet` and `scan_google_sheet`
- Polars IO plugin for lazy evaluation
- Structured exception hierarchy

---

## License

[MIT](LICENSE) © 2026 Garry (Attica-oss)

