Metadata-Version: 2.4
Name: fsspec-data
Version: 0.2.2
Summary: Arrow interchange planning for the fsspec data ecosystem
Project-URL: Repository, https://github.com/1kbgz/fsspec-data
Project-URL: Homepage, https://github.com/1kbgz/fsspec-data
Author-email: the fsspec-data authors <t.paine154@gmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: arrow,data,fsspec,interchange
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Programming Language :: Rust
Requires-Python: >=3.11
Requires-Dist: fsspec
Requires-Dist: pyarrow
Provides-Extra: develop
Requires-Dist: build; extra == 'develop'
Requires-Dist: bump-my-version; extra == 'develop'
Requires-Dist: check-dist; extra == 'develop'
Requires-Dist: cibuildwheel; extra == 'develop'
Requires-Dist: codespell; extra == 'develop'
Requires-Dist: hatch-rs; extra == 'develop'
Requires-Dist: hatchling; extra == 'develop'
Requires-Dist: mdformat; extra == 'develop'
Requires-Dist: mdformat-tables>=1; extra == 'develop'
Requires-Dist: pytest; extra == 'develop'
Requires-Dist: pytest-cov; extra == 'develop'
Requires-Dist: ruff; extra == 'develop'
Requires-Dist: twine; extra == 'develop'
Requires-Dist: ty; extra == 'develop'
Requires-Dist: uv; extra == 'develop'
Requires-Dist: wheel; extra == 'develop'
Description-Content-Type: text/markdown

# fsspec-data

Arrow interchange planning for the fsspec data ecosystem.

[![Build Status](https://github.com/1kbgz/fsspec-data/actions/workflows/build.yaml/badge.svg?branch=main&event=push)](https://github.com/1kbgz/fsspec-data/actions/workflows/build.yaml)
[![codecov](https://codecov.io/gh/1kbgz/fsspec-data/branch/main/graph/badge.svg)](https://codecov.io/gh/1kbgz/fsspec-data)
[![License](https://img.shields.io/github/license/1kbgz/fsspec-data)](https://github.com/1kbgz/fsspec-data)
[![PyPI](https://img.shields.io/pypi/v/fsspec-data.svg)](https://pypi.python.org/pypi/fsspec-data)

`fsspec-data` reconciles provided and requested tabular schemas and formats without taking
ownership of database discovery or dataframe query planning. A pure-Rust core and Python
bindings apply the same decisions to Arrow record batches and PyArrow objects.

The first release supports format-aware conversion plans, exact/projection/compatible/
coerce schema policies, registered Arrow IPC/Parquet/CSV/JSONL codecs, bounded lazy batch
iteration with cancellation, and read-only conversion through chained fsspec URLs.

## Documentation

- [Convert tabular data](https://github.com/1kbgz/fsspec-data/blob/main/docs/tutorial.md) in a
  guided first example.
- [Connect a batch producer or consumer](https://github.com/1kbgz/fsspec-data/blob/main/docs/how-to-integrate.md)
  to the interchange layer.
- [Convert a file through an fsspec chain](https://github.com/1kbgz/fsspec-data/blob/main/docs/how-to-chain-filesystems.md)
  while retaining its source filesystem.
- [Understand the interchange design](https://github.com/1kbgz/fsspec-data/blob/main/docs/explanation.md)
  and package boundaries.
- [Look up the interchange API](https://github.com/1kbgz/fsspec-data/blob/main/docs/api.md),
  including policies, codecs, and errors.
