Metadata-Version: 2.4
Name: csv-streamer
Version: 0.1.0
Summary: Stream CSV files (including from ZIP archives) as Arrow tables
Project-URL: Homepage, https://github.com/CHRISCARLON/csv-streamer
Project-URL: Repository, https://github.com/CHRISCARLON/csv-streamer
Project-URL: Issues, https://github.com/CHRISCARLON/csv-streamer/issues
Author: Chris Carlon
License-Expression: MIT
License-File: LICENSE
Keywords: arrow,csv,data,pyarrow,streaming
Requires-Python: >=3.12
Requires-Dist: pyarrow>=22.0.0
Requires-Dist: requests>=2.32.5
Description-Content-Type: text/markdown

# CSV Streamer

I find myself doing a lot of the same thing recently... and that thing is streaming large csv files over the network in batches. 

So I made a small python library using a few techniques that I know to stream csv files and yield them as batches of arrow tables for further processing/wrangling etc.

There's 4 paths and one entry point:

- stream_csv()

You can stream:

- Local CSVs
- Local CSVs in a zip file
- Remote CSVs
- Remove CSVs in a zip file
