Metadata-Version: 2.4
Name: hilcat
Version: 1.4.0
Summary: High Level Cache Tool.
Author-email: darkpeath <darkpeath@gmail.com>
Project-URL: Homepage, https://github.com/darkpeath/hilcat
Requires-Python: >=3.8
Description-Content-Type: text/x-rst
Provides-Extra: redis
Requires-Dist: redis; extra == "redis"
Provides-Extra: elasticsearch
Requires-Dist: elasticsearch; extra == "elasticsearch"
Provides-Extra: postgresql
Requires-Dist: psycopg; extra == "postgresql"
Provides-Extra: mysql
Requires-Dist: pymysql; extra == "mysql"
Provides-Extra: sqlitedict
Requires-Dist: sqlitedict; extra == "sqlitedict"
Provides-Extra: all
Requires-Dist: redis; extra == "all"
Requires-Dist: elasticsearch; extra == "all"
Requires-Dist: psycopg; extra == "all"
Requires-Dist: pymysql; extra == "all"
Requires-Dist: sqlitedict; extra == "all"

==========
HiLCaT
==========

High level key-value storage and cache tool.

Installation
============

::

  pip install hilcat

Usage
=======

Cache api is designed to determine a unique node by :code:`scope` and :code:`key`.

In some implements, :code:`scope` may be always :code:`None` and should be ignored, thus unique node determined only by :code:`key`.


Init by different backends
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

auto backend
^^^^^^^^^^^^^^^^^

create a cache based on redis

.. code-block:: python

  from hilcat import Cache

  cache = Cache.from_uri('redis://localhost:1458')

create a cache based on sqlite

.. code-block:: python

  from hilcat import Cache

  cache = Cache.from_uri('sqlite:///t.db')

in memory cache
^^^^^^^^^^^^^^^^

.. code-block:: python

  from hilcat import MemoryCache

  cache = MemoryCache()

file based cache
^^^^^^^^^^^^^^^^^^

cache text content in file

.. code-block:: python

  from hilcat import SimpleTextFileCache

  cache = SimpleTextFileCache(root_dir='cache_dir', suf='.txt')

redis
^^^^^^^^^^^^^^^^^^^^^^^

init from url

.. code-block:: python

  from hilcat import RedisCache

  cache = RedisCache(url='redis://localhost:6379')

init from host

.. code-block:: python

  from hilcat import RedisCache

  cache = RedisCache(host='localhost', port=6379)

elasticsearch
^^^^^^^^^^^^^^^^^^^

.. code-block:: python

  from hilcat import ElasticSearchCache

  cache = ElasticSearchCache(hosts=['https://localhost:9200'])

sqlite
^^^^^^^^^^^^^

.. code-block:: python

  from hilcat import SqliteCache, RelationalDbScopeConfig

  cache = SqliteCache(database=db_file, scopes=[
      RelationalDbScopeConfig(scope='a', uniq_columns=['id'], columns=['id', 'name', 'comment', 'count'],
                              column_types={'count': 'int'}),
      RelationalDbScopeConfig(scope='b', uniq_columns=['eid'], columns=['eid', 'name', 'comment', 'status'])
  ])

postgresql
^^^^^^^^^^^^^

.. code-block:: python

  from hilcat import PostgresqlCache, RelationalDbScopeConfig

  cache = PostgresqlCache(database="postgresql://postgres:123@localhost:5432/hilcat_test", scopes=[
      RelationalDbScopeConfig(scope='a', uniq_columns=['id'], columns=['id', 'name', 'comment', 'count'],
                              column_types={'count': 'int'}),
      # RelationalDbScopeConfig(scope='b', uniq_columns=['eid'], columns=['eid', 'name', 'comment', 'status'])
  ])

mysql
^^^^^^^

.. code-block:: python

  from hilcat import MysqlCache, RelationalDbScopeConfig

  cache = MysqlCache(connection=connection, scopes=[
      RelationalDbScopeConfig(scope='a', uniq_columns=['id'], columns=['id', 'name', 'comment', 'count'],
                              column_types={'id': 'varchar(50)', 'count': 'int'}),
      RelationalDbScopeConfig(scope='b', uniq_columns=['eid'], columns=['eid', 'name', 'comment', 'status'],
                              column_types={'eid': "int"})
  ])

cache api
~~~~~~~~~~~~~~~~~~~~

Assume there is a cache named :code:`cache`.

exists(key, scope=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Test if a key exists in cache for certain scope.


fetch(key, default=None, scope=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If key not exists, return default value.

.. code-block:: python

  value = cache.fetch('one', 1, scope='a')

set(key, value, scope=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: python

  cache.set('one', 1, scope='a')

update(key, value, scope=None, \*\*kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Same as method :code:`set`, but return value may diff in some implements.

get(key, func=None, scope=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If key exists, just return value stored in cache;
else if key not exists, calculate value and store to cache, the return value.

.. code-block:: python

  value = cache.get('one', lambda: 1, scope='a')

pop(key, scope=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Delete value of given key for certain scope.

scopes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Get all scopes in the cache.

May not supported for some implements.

keys(scope=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Get all keys for certain scope.

May not supported for some implements.

load(scopes=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Load scope data from persistence storage.

Some implements may have no persistence storage, thus this method do nothing.

backup(scopes=None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Save scope data to persistence storage.

Some implements may have no persistence storage, thus this method do nothing.

Decorate a function
~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    import collections
    from hilcat import SqliteCache, RelationalDbScopeConfig

    db_file = "decorator.db"
    cache = SqliteCache(database=db_file, scopes=[
        RelationalDbScopeConfig(scope='f1', uniq_columns=['x'], columns=['y']),
        RelationalDbScopeConfig(scope='f3', uniq_columns=['key'], columns=['key', 'value'])
    ])

    c1 = collections.Counter()
    @cache(scope="f1")
    def f1(x: int):
        c1[x] += 1
        return x + 1 + c1[x]

    c2 = collections.Counter()
    def f2(x: int):
        c2[x] += 1
        return x + 1 + c2[x]

    def make_key(x: int, y: int):
        return '-'.join(map(str, [x, y]))
    c3 = collections.Counter()
    @cache(scope="f3", make_key_func=make_key)
    def f3(x: int, y: int):
        c3[(x, y)] += 1
        return {
            "key": make_key(x, y),
            "value": x + y + c3[(x, y)],
        }

    # with cache, same result
    assert f1(1) == 3
    assert f1(1) == 3

    # without cache, different result
    assert f2(1) == 3
    assert f2(1) == 4

    assert f3(1, 2) == {"key": "1-2", "value": 4}
    assert f3(1, 2) == {"key": "1-2", "value": 4}

