Metadata-Version: 2.5
Name: vruksha
Version: 0.0.2
Summary: an entity graph over a litesearch store: extraction, resolution, and a PageRank search leg
Project-URL: Repository, https://github.com/vedicreader/vruksha
Project-URL: Documentation, https://vedicreader.github.io/vruksha/
Author-email: Karthik <karthik.rajgopal@hotmail.com>
License: Apache-2.0
License-File: LICENSE
Keywords: entity resolution,knowledge graph,nbdev,pagerank,search,sqlite
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Text Processing :: Indexing
Requires-Python: >=3.10
Requires-Dist: fastcore>=2.2.15
Requires-Dist: litesearch>=0.1.33
Requires-Dist: yake>=0.7.3
Description-Content-Type: text/markdown

# vruksha


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

Four problems, and none of them needs a model.

| problem | how it is solved |
|----|----|
| what the entities are | AST symbols for code, yake keyphrases for prose |
| which mentions are the same thing | embeddings propose, a lexical guard decides |
| what connects to what | PMI over co-occurrence windows |
| what to do with the graph | personalised PageRank, fused as a third search leg |

``` python
from litesearch import database
from vruksha import build_graph, resolve_entities

db = database('corpus.db')
build_graph(db, rows, emb_fn=enc)     # entities and co-occurrence edges
resolve_entities(db)                  # `hnsw` and `HNSW index` become one node
db.graph_search('how does resolution work', qemb, graph_w=0.5)
```

## The lexical guard

Embedding similarity alone merges `python 3.11` into `python 3.12`. [`_lex_ok`](https://vedicreader.github.io/vruksha/entities.html#_lex_ok) requires token
overlap, matching digits and a matching acronym before a merge goes through.

``` python
from vruksha.entities import _lex_ok

_lex_ok('usearch', 'usearch index'), _lex_ok('python 3.11', 'python 3.12')
```

## When to turn the search leg on

Measured against plain hybrid search:

- **Regulation and legal text: a loss.** p_mrr 0.8170 for plain hybrid against 0.7395, 0.6859 and
  0.6463 at `graph_w` 0.25, 0.5 and 1.0, at two to four times the latency.
- **Papers and prose: a win.** Better in seven of nine paired-bootstrap comparisons, +0.0387
  target MRR on arXiv at `graph_w=1.0`.

So `graph_search` is opt-in by name and off by default. Turn it on for a corpus whose entities
carry meaning, and raise `graph_w` towards 1.0 when you do.

## Install

``` sh
pip install vruksha
```
