Metadata-Version: 2.4
Name: fastnumparse
Version: 0.3.0
Summary: Fast number parsing for NumPy, powered by C++.
Keywords: numpy,number-parsing,parser
Author: AndrewChan2022
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: C++
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Project-URL: Homepage, https://github.com/AndrewChan2022/fastnumparse
Project-URL: Repository, https://github.com/AndrewChan2022/fastnumparse
Project-URL: Issues, https://github.com/AndrewChan2022/fastnumparse/issues
Requires-Python: <3.15,>=3.10
Requires-Dist: numpy>=1.23
Provides-Extra: dev
Requires-Dist: build>=1.2.2; extra == "dev"
Requires-Dist: cibuildwheel<5,>=4.2; extra == "dev"
Requires-Dist: mypy>=1.11; extra == "dev"
Requires-Dist: py7zr<2,>=1.0; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: twine>=6; extra == "dev"
Description-Content-Type: text/markdown


# fastnumparse

Fast number parsing for NumPy, powered by C++.


## install

requires-python = ">=3.10,<3.15"

```bash
pip install fastnumparse
```

## usage

set parallel thread count:

```pyton
# max 16 threads
fnp.set_max_threads(16)

# max threads same as cpu core number
fnp.set_max_threads(0)
```



for csv-like data, with row * col,  with sep=" " and comment ="#"

```txt
1.694992566108704e+01 4.373334741592407e+01 1.491762180328369e+02  # comment
1.875837993621826e+01 4.405829811096191e+01 1.520713310241699e+02
1.766352510452271e+01 4.732764196395874e+01 1.513219413757324e+02
```

```python
import fastnumparse as fnp

fnp.set_max_threads(16)

path = "csv_data.txt"
raw_buffer = open(path, "rb").read()
values, _ = fnp.from_string_buffer_csv(
    raw_buffer,
    offset=0,
    dtype=np.float64,
    comment="#",
    max_rows = 3,
    column_count=3,
    ndmin=2,            # 2: return 2d array, 0~1: return 1d array
)
```


for noncsv data, each row unkown number count, with sep=" " and comment ="#", and with an end char
```txt
15 780721 655221 
955795 127489 609646 896903 101275 343792 898984
699394 898633 781093 1361969 631076 899312 898945 899053 780575 149987
}
```


```python
import fastnumparse as fnp

fnp.set_max_threads(16)

path = "noncsv_data.txt"
raw_buffer = open(path, "rb").read()
values, _ = fnp.from_string_buffer_noncsv(
    raw_buffer,
    offset=0,
    dtype=np.int64,
    comment="#",
    end_char="}",
    nelement=20,
)
```


## Benchmark

### windows

Naive Python vs. NumPy vs. fastnumparse on Windows. Lower is better.

<p align="center">
  <img src="docs/images/benchmark-windows-csv-float-244768x3.png" alt="Windows float CSV benchmark" width="75%"><br>
  <img src="docs/images/benchmark-windows-csv-int-2854577x4.png" alt="Windows integer CSV benchmark" width="75%"><br>
  <img src="docs/images/benchmark-windows-noncsv-int-244761.png" alt="Windows integer non-CSV benchmark" width="75%">
</p>

Tested on Windows 11 with an Intel Core i7-11800H, Python 3.13.2,
NumPy 2.4.2, and fastnumparse 0.2.0 using 16 threads. Each input was loaded
into memory before timing. The chart shows one call with no warm-up; file I/O
is excluded. Each bar is labeled with its raw time in milliseconds.

### Linux

Naive Python vs. NumPy vs. fastnumparse on Linux. Lower is better.

<p align="center">
  <img src="docs/images/benchmark-linux-csv-float-244768x3.png" alt="Linux float CSV benchmark" width="75%"><br>
  <img src="docs/images/benchmark-linux-csv-int-2854577x4.png" alt="Linux integer CSV benchmark" width="75%"><br>
  <img src="docs/images/benchmark-linux-noncsv-int-244761.png" alt="Linux integer non-CSV benchmark" width="75%">
</p>

Each input was loaded into memory before timing. The chart shows one call with
no warm-up; file I/O is excluded. Each bar is labeled with its raw time in
milliseconds.




## Development

See the [development, build, and publishing guide](docs/development.md).

## Algorithm 

The algorithm how to speedup at: [algorithm](docs/algorithm.md).

## limited

- comment only support # now
- delimiter only support space now

## Acknowledgements

- [nanothread](https://github.com/mitsuba-renderer/nanothread) for light tbb style parallel_for
- [fast_float](https://github.com/fastfloat/fast_float) for fast string to float
- [pybind11](https://github.com/pybind/pybind11) for python binding
