Metadata-Version: 2.1
Name: dqp
Version: 0.1.1
Summary: Cache iterator values to disk using msgpack
Home-page: https://bneijt.github.io/dqp/
License: Apache-2.0
Author: Bram Neijt
Author-email: bram@neijt.nl
Requires-Python: >=3.9,<4.0
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: msgpack (>=1.0.7,<2.0.0)
Project-URL: Repository, https://github.com/bneijt/dqp
Description-Content-Type: text/markdown

# Disk Queue Processing (dqp)

This library makes it easy to serialize Python to disk using msgpack.

The library contains two modules:

- `dqp.disk_cache`: to easily read/write data to disk and support caching iterators to disk.
- `dqp.disk_queue`: an approach to communicating lists of objects between runs with index/offset metadata.

## Example of disk_cache

```python
from typing import Iterator
from dqp.disk_cache import cached_iter
import time

@cached_iter()
def expensive_iter() -> Iterator[int]:
    for idx in range(5):
        time.sleep(1)
        yield idx

def main():
    #For replay, drop the cache
    expensive_iter.cache_clear()

    # First time it takes seconds
    for i in expensive_iter():
        print(time.asctime(), i)

    # Other times the cache is already there
    print(time.asctime(), list(expensive_iter()))
    print(time.asctime(), list(expensive_iter()))
    print(time.asctime(), list(expensive_iter()))

if __name__ == "__main__":
    main()

```

