Metadata-Version: 2.4
Name: polars-hash
Version: 0.9.1
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Requires-Dist: polars>=1.36.1
Summary: Stable non-cryptographic and cryptographic hashing functions for Polars
Author-email: Ion Koutsours <15728914+ion-elgreco@users.noreply.github.com>
Requires-Python: >=3.10
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Bug Tracker, https://github.com/ion-elgreco/polars-hash/issues
Project-URL: Homepage, https://github.com/ion-elgreco/polars-hash

This plugin provides stable hashing functionality across different polars versions.

📖 **[Documentation](https://ion-elgreco.github.io/polars-hash/)** — every expression,
its input and output types, and its arguments.

## Examples
### Cryptographic Hashers

```python
import polars as pl
import polars_hash as plh

df = pl.DataFrame({
    "foo":["hello_world"]
})

result = df.select(plh.col('foo').chash.sha256())

print(result)

┌──────────────────────────────────────────────────────────────────┐
│ foo                                                              │
│ ---                                                              │
│ str                                                              │
╞══════════════════════════════════════════════════════════════════╡
│ 35072c1ae546350e0bfa7ab11d49dc6f129e72ccd57ec7eb671225bbd197c8f1 │
└──────────────────────────────────────────────────────────────────┘
```

### Non-cryptographic Hashers
```python
df = pl.DataFrame({
    "foo":["hello_world"]
})

result = df.select(plh.col('foo').nchash.wyhash())
print(result)
┌──────────────────────┐
│ foo                  │
│ ---                  │
│ u64                  │
╞══════════════════════╡
│ 16737367591072095403 │
└──────────────────────┘

result = df.select(plh.col('foo').nchash.farmhash64())
print(result)
┌──────────────────────┐
│ foo                  │
│ ---                  │
│ u64                  │
╞══════════════════════╡
│ 15605398435621216523 │
└──────────────────────┘

result = df.select(plh.col('foo').nchash.farmhash32())
print(result)
┌────────────┐
│ foo        │
│ ---        │
│ u32        │
╞════════════╡
│ 1719156559 │
└────────────┘

result = df.select(plh.col('foo').nchash.cityhash128())
print(result)
┌─────────────────────────────────────────┐
│ foo                                     │
│ ---                                     │
│ u128                                    │
╞═════════════════════════════════════════╡
│ 133423608296839006301901834072762183026 │
└─────────────────────────────────────────┘

result = df.select(plh.col('foo').nchash.gxhash64())
print(result)
┌─────────────────────┐
│ foo                 │
│ ---                 │
│ u64                 │
╞═════════════════════╡
│ 2180020304351407825 │
└─────────────────────┘
```

`cityhash32()` and `cityhash64()` return the values printed above for `farmhash32()`
and `farmhash64()`. That is expected: FarmHash reuses CityHash for short input, and
`hello_world` is 11 bytes. See
[the CityHash reference](https://ion-elgreco.github.io/polars-hash/latest/api-reference/non-cryptographic/#cityhash32).

The GxHash expressions need a CPU with AES instructions and have no software fallback.
Every x86, x86-64 and aarch64 wheel is built for them; there are no `linux-armv7` or
`linux-ppc64le` wheels from 0.8.0 on, because GxHash cannot be built for either. See
[the GxHash reference](https://ion-elgreco.github.io/polars-hash/latest/api-reference/non-cryptographic/#gxhash32).

### Geo Hashers
```python
df = pl.DataFrame(
    {"coord": [{"longitude": -120.6623, "latitude": 35.3003}]},
    schema={
        "coord": pl.Struct(
            [pl.Field("longitude", pl.Float64), pl.Field("latitude", pl.Float64)]
        ),
    },
)

df.with_columns(
    plh.col('coord').geohash.from_coords().alias('geohash')
)
shape: (1, 2)
┌─────────────────────┬──────────────┐
│ coord               ┆ geohash      │
│ ---                 ┆ ---          │
│ struct[2]           ┆ str          │
╞═════════════════════╪══════════════╡
│ {-120.6623,35.3003} ┆ 9q60y60rhsgg │
└─────────────────────┴──────────────┘


pl.select(pl.lit('9q60y60rhs').geohash.to_coords().alias('coordinates'))
shape: (1, 1)
┌───────────────────────┐
│ coordinates           │
│ ---                   │
│ struct[2]             │
╞═══════════════════════╡
│ {-120.6623,35.300298} │
└───────────────────────┘
```

### H3 Spatial Index
```python
df = pl.DataFrame(
    {"coord": [{"longitude": -120.6623, "latitude": 35.3003}]},
    schema={
        "coord": pl.Struct(
            [pl.Field("longitude", pl.Float64), pl.Field("latitude", pl.Float64)]
        ),
    },
)

df.with_columns(
    plh.col('coord').h3.from_coords().alias('h3')
)
shape: (1, 2)
┌─────────────────────┬─────────────────┐
│ coord               ┆ h3              │
│ ---                 ┆ ---             │
│ struct[2]           ┆ str             │
╞═════════════════════╪═════════════════╡
│ {-120.6623,35.3003} ┆ 8c29adc423821ff │
└─────────────────────┴─────────────────┘
```


### Time Hasher

Bins timestamps into variable-precision sliding windows of time, so rows that
fall in the same window share a hash. Timestamps must lie between 1970-01-01 and
2098-01-01. A higher precision means a shorter window: 10 covers about 4 seconds,
8 about 4 minutes.

Precision may be 1 to 32, but past about 18 the hash stops changing for present-day
timestamps and the extra characters are padding. The exact point depends on the date:
timestamps close to 1970 keep splitting to about 21, far-future ones run out sooner.

```python
from datetime import datetime

df = pl.DataFrame({"datetime": [datetime(2017, 2, 21, 20, 15, 13)]})

df.with_columns(
    plh.col('datetime').timehash.from_datetime().alias('timehash')
)
shape: (1, 2)
┌─────────────────────┬────────────┐
│ datetime            ┆ timehash   │
│ ---                 ┆ ---        │
│ datetime[μs]        ┆ str        │
╞═════════════════════╪════════════╡
│ 2017-02-21 20:15:13 ┆ afcccc0e1b │
└─────────────────────┴────────────┘


pl.select(pl.lit('afcccc0e1b').timehash.to_datetime().alias('datetime'))
shape: (1, 1)
┌────────────────────────────────┐
│ datetime                       │
│ ---                            │
│ datetime[μs, UTC]              │
╞════════════════════════════════╡
│ 2017-02-21 20:15:11.292315 UTC │
└────────────────────────────────┘


pl.select(pl.lit('afcccc0e1b').timehash.neighbors().alias('neighbors'))
shape: (1, 1)
┌─────────────────────────────┐
│ neighbors                   │
│ ---                         │
│ struct[2]                   │
╞═════════════════════════════╡
│ {"afcccc0e1a","afcccc0e1c"} │
└─────────────────────────────┘
```

## Create hash from multiple columns
```python
df = pl.DataFrame({"foo": ["hello_world"], "bar": ["today"]})

result = df.select(plh.concat_str("foo", "bar").chash.sha256())
```

## Hash a whole row

`hash_rows` gives each row bytes that no other row can make, for all column types.
Any hasher then reads those bytes.

```python
df = pl.DataFrame(
    {"foo": ["hello_world"], "bar": [42], "baz": [[1, 2, 3]], "qux": [{"a": 1}]}
)

df.select(plh.hash_rows(pl.all()).chash.sha2_256())
shape: (1, 1)
┌──────────────────────────────────────────────────────────────────┐
│ foo                                                              │
│ ---                                                              │
│ str                                                              │
╞══════════════════════════════════════════════════════════════════╡
│ 9055866af8d3c113e0a8fdb729ce8e6fa67ed5f6f51efa8235a588e88ea972f4 │
└──────────────────────────────────────────────────────────────────┘
```

The encoder reads the meaning of a value, not the polars storage of it. An `Int32` and
the `Int64` next to it make the same hash. A `Datetime` in milliseconds and the same
time in nanoseconds also make the same hash, and a `Categorical` makes the hash of its
string. The encoder does not read the column names. Therefore a new name keeps the
hash, but a new order does not. The
[reference](https://ion-elgreco.github.io/polars-hash/latest/api-reference/rows/)
gives all the rules and the byte layout of version 1, which does not change.

