Metadata-Version: 2.4
Name: teadata
Version: 0.0.118
Summary: Pythonic data engine for Texas education datasets
Author-email: Alejandro Peña <adpena@gmail.com>
License: Business Source License 1.1
        
        Parameters
        Licensor: Alejandro Peña
        Licensed Work: teadata
        Additional Use Grant: You may use, copy, modify, and distribute this software for personal, educational, or non-commercial research purposes.
        
        Terms
        The Licensor hereby grants you the right to copy, modify, create derivative works, redistribute, and make non-production use of the Licensed Work. The licensee is prohibited from using the Licensed Work in production or in any commercial product or service without obtaining a separate commercial license from the Licensor.
        
        For clarity, “production use” includes any use of the Licensed Work, or a modified version of it, in a way that provides a service or product to others for compensation or in support of business operations.
        
        THE LICENSED WORK IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE, OR NON-INFRINGEMENT. IN NO EVENT SHALL THE LICENSOR BE LIABLE FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT, OR OTHERWISE, ARISING FROM, OUT OF, OR IN CONNECTION WITH THE LICENSED WORK OR THE USE OR OTHER DEALINGS IN THE LICENSED WORK.
Project-URL: Homepage, https://github.com/adpena/teadata
Project-URL: Bug Tracker, https://github.com/adpena/teadata/issues
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Scientific/Engineering :: GIS
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: python-dateutil>=2.8.2
Requires-Dist: numpy>=1.26
Requires-Dist: pandas>=2.1
Requires-Dist: shapely>=2.0
Requires-Dist: geopandas>=0.14
Requires-Dist: pyogrio>=0.7
Requires-Dist: pyyaml>=6.0
Requires-Dist: openpyxl>=3.1
Requires-Dist: duckdb>=0.10
Requires-Dist: pyarrow>=14.0
Requires-Dist: packaging>=23.2
Requires-Dist: scipy>=1.10
Requires-Dist: platformdirs>=3.7
Requires-Dist: tomli>=2.0
Provides-Extra: speedups
Requires-Dist: pyogrio>=0.7; extra == "speedups"
Requires-Dist: polars>=1.0; extra == "speedups"
Requires-Dist: pygeos; python_version < "3.11" and extra == "speedups"
Provides-Extra: dev
Requires-Dist: ruff>=0.5; extra == "dev"
Requires-Dist: ty>=0.0.11; extra == "dev"
Requires-Dist: pytest>=7.4; extra == "dev"
Requires-Dist: pytest-cov>=4.1; extra == "dev"
Requires-Dist: pip-tools>=7.4; extra == "dev"
Requires-Dist: pip-audit>=2.7; extra == "dev"
Requires-Dist: deptry>=0.16; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.24; extra == "docs"
Provides-Extra: notebooks
Requires-Dist: jupyterlab>=4.0; extra == "notebooks"
Requires-Dist: ipython>=8.0; extra == "notebooks"
Requires-Dist: ipywidgets>=8.0; extra == "notebooks"
Requires-Dist: ipyleaflet>=0.18; extra == "notebooks"
Requires-Dist: itables>=1.6; extra == "notebooks"
Requires-Dist: google-auth>=2.0; extra == "notebooks"
Requires-Dist: google-api-python-client>=2.0; extra == "notebooks"
Provides-Extra: database
Requires-Dist: sqlalchemy>=2.0; extra == "database"
Requires-Dist: psycopg[binary]>=3.1; extra == "database"
Provides-Extra: scripts
Requires-Dist: requests>=2.31; extra == "scripts"
Requires-Dist: tqdm>=4.66; extra == "scripts"
Requires-Dist: geopy>=2.4; extra == "scripts"
Requires-Dist: aiohttp>=3.9; extra == "scripts"
Dynamic: license-file

# TEA Data Engine (`teadata`)

`teadata` is a snapshot-first Python engine for Texas education data.
It provides:

- `District` and `Campus` domain models
- a fluent query DSL using `>>`
- geospatial lookups (nearest charter, campuses in district boundaries, private-school overlap)
- config-driven enrichment from TAPR, accountability, transfers, PEIMS financials, and closure datasets
- sidecar sqlite stores for fast boundary/map/entity lookup

## Installation

### PyPI

```bash
pip install teadata
```

### Development (recommended)

```bash
git clone https://github.com/adpena/teadata.git
cd teadata
uv sync --all-extras
```

## Quick Start

```python
from teadata import DataEngine

# Preferred runtime path: load the latest discovered snapshot.
engine = DataEngine.from_snapshot(search=True)

# District lookup by district number, campus number, or name.
aldine = engine.get_district("101902")
print(aldine.name)

# Campuses physically inside district boundaries.
for campus in aldine.campuses[:5]:
    print(campus.name, campus.campus_number)
```

## Public API Surface

Primary imports:

```python
from teadata import DataEngine, District, Campus
```

Core behaviors:

- `DataEngine.from_snapshot(...)` supports `.pkl` and `.pkl.gz` snapshots and multiple payload shapes.
- Snapshot discovery checks explicit paths, env vars, package `.cache`, and parent `.cache` directories.
- `District` and `Campus` support dynamic metadata attributes through `meta`.
- `Campus.to_dict()` always includes `percent_enrollment_change` (numeric when available, otherwise `"N/A"`).

## Snapshot and Asset Behavior

`teadata` is intentionally cache-first.

Artifacts typically used at runtime:

- `repo_*.pkl` / `repo_*.pkl.gz` (engine snapshot)
- `boundaries_*.sqlite` (boundary WKB sidecar)
- `map_payloads_*.sqlite` (map payload sidecar)
- `entities_*.sqlite` (entity lookup sidecar)

If snapshot/store files are Git LFS pointers or missing locally, runtime asset resolvers can fetch real files when URL env vars are provided.

### Environment Variables

- `TEADATA_SNAPSHOT`: explicit snapshot path.
- `TEADATA_SNAPSHOT_URL`: URL used when snapshot candidate is missing or a Git LFS pointer.
- `TEADATA_BOUNDARY_STORE`: explicit boundary sqlite path.
- `TEADATA_BOUNDARY_STORE_URL`: URL fallback for boundary store.
- `TEADATA_MAP_STORE`: explicit map sqlite path.
- `TEADATA_MAP_STORE_URL`: URL fallback for map store.
- `TEADATA_ENTITY_STORE`: explicit entity sqlite path.
- `TEADATA_ENTITY_STORE_URL`: URL fallback for entity store.
- `TEADATA_ASSET_CACHE_DIR`: override cache directory used for downloaded assets.
- `TEADATA_DISABLE_INDEXES`: disable default spatial acceleration indexes.
- `TEADATA_LOG_MEMORY`: enable memory snapshot logging.

## Query DSL

`DataEngine` and `Query` chains use `>>`.

```python
# Resolve district then expand to district-operated campuses.
q = engine >> ("district", "ALDINE ISD") >> ("campuses_in",)

# Filter, sort, and take.
top = (
    q
    >> ("filter", lambda c: (c.enrollment or 0) > 1000)
    >> ("sort", lambda c: c.enrollment or 0, True)
    >> ("take", 10)
)

rows = top.to_df(columns=["name", "campus_number", "enrollment"])
```

Supported lookup semantics include:

- case-insensitive district and campus name matching
- wildcard patterns (`*`, `?`, SQL-like `%`/`_`)
- normalized district number handling (for example `"123"` and `"'000123"`)

Spatial and transfer helpers include:

- nearest-campus/nearest-charter queries
- `nearest_charter_same_type(...)`
- transfer graph methods such as `transfers_out(...)` / `transfers_in(...)`

## Enrichment Pipeline

`teadata/enrichment` provides registered enrichers for district and campus datasets.

Included enrichers cover:

- district accountability and district TAPR profile data
- campus accountability, TAPR profile/historical enrollment, PEIMS financials
- planned closure overlays
- charter network augmentation

Pipeline behavior is fault-tolerant by design: dataset-level failures are generally logged and do not hard-stop the full build.

## Data Build Pipeline

`teadata/load_data.py` builds a full `DataEngine` and updates cached artifacts.

```bash
uv run python -m teadata.load_data
```

At a high level, it:

1. resolves year-aware source paths from `teadata/teadata_sources.yaml`
2. warm-loads compatible snapshot cache when signatures match
3. otherwise builds districts/campuses from spatial files
4. applies enrichment datasets
5. writes snapshot + sqlite sidecars back to `.cache/`

## Config and CLI (`teadata-config`)

`teadata/teadata_config.py` provides YAML/TOML config loading, year resolution, schema checks, and dataset joins.

CLI entrypoint:

```bash
uv run teadata-config --help
```

Subcommands:

- `init <out.yaml>`
- `resolve <cfg> <section> <dataset> <year>`
- `report <cfg> [--json] [--min N] [--max N]`
- `join <cfg> <year> [--datasets a,b,c] [--parquet out.parquet] [--duckdb out.duckdb --table t]`

## Testing

```bash
uv run pytest
```

Current tests cover:

- snapshot gzip and fallback loading
- query DSL semantics and chaining
- nearest charter behavior and transfer grouping
- store discovery and asset-cache behavior
- entity serialization invariants (`percent_enrollment_change`)

## PyPI Size Limits and Current Packaging Status

PyPI defaults currently documented at:

- per-file upload limit: `100 MB`
- total project limit: `10 GB`

Reference: <https://docs.pypi.org/project-management/storage-limits/>

Current `teadata` release artifacts for `0.0.118` are above the per-file limit:

- wheel: `dist/teadata-0.0.118-py3-none-any.whl` about `448 MB`
- sdist: `dist/teadata-0.0.118.tar.gz` about `446 MB`

These exceed the default 100 MB file cap because large `.cache` snapshot/store artifacts are packaged into both distributions.

## Release Policy

- Versioning uses thousandths-place tags (`v0.0.101`, `v0.0.102`, ...).
- Keep only the 3 most recent release tags/assets.

## License

Business Source License 1.1. See [LICENSE](LICENSE).
