Metadata-Version: 2.4
Name: jawntmap
Version: 1.1.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Rust
Classifier: Topic :: Database
Classifier: Topic :: Text Processing
Summary: Fast entity/value resolution and NL2SQL grounding
Keywords: entity-resolution,entity-matching,nl2sql,canonicalization,fuzzy-matching
Author: Vincent Berry
License: MIT
Requires-Python: >=3.9, <3.14
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Documentation, https://github.com/vaberry/jawntmap#readme
Project-URL: Homepage, https://github.com/vaberry/jawntmap
Project-URL: Issues, https://github.com/vaberry/jawntmap/issues
Project-URL: Repository, https://github.com/vaberry/jawntmap

# Jawntmap

Jawntmap is a fast entity/value resolution layer for NL2SQL and structured-data
search. It maps messy user text such as aliases, abbreviations, typos, compact
codes, and business-period phrases onto canonical database values.

This Python package exposes the core field-level resolver:

```python
from jawntmap import EntityResolver

resolver = EntityResolver(
    ["Apple Inc.", "Microsoft Corporation", "Consulting Services Q1"],
    aliases={
        "Apple Inc.": ["Apple", "AAPL"],
        "Microsoft Corporation": ["MSFT", "Microsoft"],
    },
)

print(resolver.resolve("microsft", top_n=3))
```

It can also build a dataset catalog directly from SQLite or CSV. The catalog
profiles tables, skips fields that look numeric/date/identifier/free-text, mines
small sibling code fields as aliases for name-like fields, extracts
collision-free domain aliases from skipped `website`, `domain`, and
`contact_email` siblings, and builds
field-aware resolvers for the remaining entity, category, and code fields. CSV
cataloging treats one file as one logical table and infers coarse
numeric/date/boolean/text column types before using the same resolver path.
Python can also catalog an iterable of dict-like records directly with
`jawntmap.index_records`, using in-memory schema introspection, exact row/null/
empty diagnostics, distinct value counts, sibling alias mining, and aliases
embedded inside canonical values such as parentheticals, `aka`, and `d/b/a`.
Pass `normalize_names=True` when messy table names or dict keys should become
stable identifiers such as `vendor_export_2026.vendor_name`.
CSV and records cataloging cap exact distinct-count diagnostics at one million
retained distinct values by default; pass `exact_distinct_limit=...`, or `0` to
disable the cap, when you need to tune that diagnostic for very large fields.
Fields proven to contain exactly one non-empty distinct value stay visible in
catalog metadata but are skipped as generated resolvers with a
`constant_value` reason.
Field dictionaries expose profile stats such as sampled row-frequency mass,
dominant-value frequency, singleton-value ratio, row/null/empty counts, null
and empty rates, and uniqueness ratio when the backend can compute them;
SQLite, CSV, MySQL, and PostgreSQL catalogs currently provide those diagnostics.
Fully sampled resolver-worthy fields whose top value reaches 80% of sampled
rows also include a `dominant_value` reason while remaining searchable.
Resolver-worthy fields populated in at most 20% of rows include a non-skipping
`sparse_values` reason when exact row diagnostics are available.
They also expose normalized-value collision counts so callers can see when
distinct stored values collapse to the same normalized surface.
They also expose token-signature collision counts so callers can see when
distinct stored values have the same sorted normalized tokens, such as
`Jane Smith` and `Smith Jane`. Small text entity resolvers use the same
order-insensitive token signature to resolve mentions such as `smith jane` to a
stored `Jane Smith` when the field evidence supports it.
They also expose alias-collision counts so callers can see when the same mined
or metadata alias points at multiple canonical values in one field.
They also expose `identifier_like_ratio`, `location_like_ratio`,
`entity_like_ratio`, `person_like_ratio`, `named_thing_like_ratio`, `boolean_like_ratio`,
`numeric_like_ratio`, and `temporal_like_ratio`; email, URL, phone, IP, UUID,
hash, and token-shaped fields are treated as skipped identifiers, tiny `yes/no`
or `true/false` text fields are treated as skipped booleans, TEXT-stored
numeric/currency/percent measures plus delimited date/timestamp fields are
treated as skipped numerics or temporals, country/state/province-heavy text
fields can profile as locations even when not named `country`, legal/org-shaped
fields such as `issuer` can profile as entity-name fields, and
personal-name-heavy role fields such as `assignee` can do the same. Named
product/service/procedure-like fields such as `offering` can also profile as
entity-name fields and mine sibling code aliases such as `item_code ->
offering`. Skipped web identifier siblings can contribute safe domain aliases
to those entity-name fields, so `company abc.xyz` can resolve to
`Alphabet Inc.` without indexing URL/email columns as fuzzy text. Clearly named
standalone location-code fields such as `country_code`, `state_code`, and
`province_code` can also generate collision-free labels, so `country code
deutschland` can resolve a stored `DE` even without a sibling country-name
column. Location fields also receive dataset-grounded country exonym aliases
and state/province abbreviation aliases, so catalogs with stored `Germany` or
`California` values can resolve `Deutschland` or `CA`.
They also receive generated country demonym aliases, so `Germany` can resolve
from `german` with `generated_location_demonym_alias` provenance.
Entity-name fields also receive conservative generated initialism aliases for
collision-free canonical values, so `Premium Support Plan` can resolve `PSP`
with `generated_initialism` provenance even without a sibling code column.
Named product/service/procedure-like fields receive conservative generated
full-value inflection aliases, so `Premium Support Plan` can resolve from
`premium support plans` with `generated_inflection_alias` provenance when that
plural surface is collision-free inside the field.
Common schema-role names such as `supplier`, `client`, `facility`, `provider`,
`diagnosis`, `drug`, `fund`, and `venue` are also profiled as entity-name
fields even when their sampled values are short labels, letting paired code
fields mine aliases such as `supplier -> supplier_code` from row data.
Status-like category fields receive conservative generated status aliases, so
`active` can resolve from `enabled` with `generated_status_alias` provenance
when that alias is collision-free inside the field.
Organization-like entity-name fields receive conservative generated short-name
aliases from legal/business suffix stripping, so `Apple Inc.` can resolve from
`apple` with `generated_organization_alias` provenance when the short name is
collision-free inside the field.
Person-like entity-name fields receive conservative generated first-last,
surname, and first-initial surname aliases, so `Stephen A. Curry` can resolve
from `stephen curry` and `Jane Doe` can resolve from `doe` or `j doe` with
`generated_person_alias` provenance when those aliases are collision-free.
They also receive conservative first-name nickname aliases, so `Robert Smith`
can resolve from `bob smith` with `generated_person_nickname_alias`
provenance.
Optional alias metadata can attach alternate names, nicknames, previous names,
and valid-year alias windows to generated catalog resolvers. Catalog-wide
ranking prefers explicit ticker/code contexts for code fields, entity contexts
for name aliases, and uppercase or mixed-case alias-shaped mentions for
canonical entity-name values while preserving lower-case exact code lookups.
Results include deterministic ranking metadata so downstream callers can see
whether a hit came from a canonical value or an alias, and ambiguity-group
metadata so planners can detect when table/field context is needed. Results
also use query role cues such as `opponent`, `against`, `home`, `away`,
`venue`, `vendor`, `customer`, `issuer`, and `assignee` to boost matching
table/field roles while preserving `field_role`, `same_value`, and `score_tie`
ambiguity labels. Results
try clean context-trimmed multi-token spans before broader span groups, so a
phrase such as `customer Apex Analytics Customer 000000` can test the exact
value span before fuzzy lookup on the surrounding scope word; grammar bridges
such as `from` keep the normal fallback order. Three-token-or-longer exact
canonical value spans also use an exact preflight across saved field resolvers
before broad fuzzy search, preserving duplicate exact field candidates while
avoiding near-neighbor alternatives, including persisted fields outside the
compact catalog sample. Multi-token spans with a strong
serial/code anchor also try saved field resolvers before broad fuzzy retrieval,
and short-circuit only when the non-anchor words support the candidate. Results
also expose `preferred_table_match`, `preferred_field_match`,
`preferred_semantic_type_match`, and `context_preference_boost`, making soft
table/field/semantic hints auditable. When `preferred_table` is connected to a
candidate table through declared foreign keys, results also expose
`preferred_table_join_match`, `preferred_table_join_hops`, and
`preferred_table_join_path` so planners can see that a value should be grounded
through a join instead of an exact table match. Results also expose
`query_context_*` fields that explain soft boosts inferred directly from the
natural-language query, such as table/field-name cues, role cues, and semantic
cues like `ticker`, `country`, or `status`. When a planner already knows the expected
entity class, pass `semantic_type="location"` or `semantic_type="code"` to
hard-scope lookup to fields of that type; unlike
`preferred_semantic_type`, this hides alternatives outside the requested
semantic type. Calls can also pass previous resolved candidates or context
dicts through `context_values`; returned
`context_value_*` diagnostics explain same-table, joined-table, same-field,
semantic, and exact-value support from nearby grounded values, with
`context_value_table_join_paths` showing declared FK paths when the support
comes from a related table. Candidate results include compact field
diagnostics such as `field_readiness_status`, `field_readiness_reasons`,
`field_reasons`,
`field_resolver_coverage_status`, `field_truncated`, `field_high_cardinality`, `field_max_value_frequency`,
`field_singleton_value_ratio`, `field_row_count`, `field_empty_rate`,
`field_uniqueness_ratio`, `field_avg_tokens`, `field_code_like_ratio`,
`field_identifier_like_ratio`, `field_location_like_ratio`,
`field_entity_like_ratio`, `field_person_like_ratio`,
`field_named_thing_like_ratio`,
`field_normalized_collision_count`, `field_alias_collision_count`,
`field_token_signature_collision_count`,
`sampled_value_count`, and
`sampled_value_frequency`. Alias-backed candidates
also expose `matched_alias_evidence`, `matched_alias_source_field`,
`matched_alias_count`, and `alias_evidence_boost` so callers can tell whether
the alias came from a sibling field, metadata, built-in alias data, a generated
organization short name, a
generated full-value inflection, a generated initialism, or a generated
person-name alias, and whether repeated row-paired evidence helped ranking:

```python
import jawntmap

catalog = jawntmap.index_sqlite(
    "shop.db",
    sample_limit=1000,
    resolver_sample_limit=25_000,
    aliases=[
        {
            "table": "companies",
            "column": "name",
            "canonical": "Alphabet Inc.",
            "aliases": ["Google", "Alphabet Class A"],
        }
    ],
)

print([field["path"] for field in catalog.eligible_fields()])
print(catalog.field("companies", "name", include_values=True)["value_aliases"])

print(catalog.resolve("apple", table="orders", field="vendor_name", top_n=3))
print(catalog.resolve_anywhere("active orders", table="orders", top_n=5))
match = catalog.resolve_anywhere("ticker aapl", top_n=5)[0]
print(match["value"], match["field"], match["match_kind"], match["ranking_score"])
print(match["matched_alias"], match["matched_alias_source_field"])
print(match["query_context_cues"], match["query_context_boost"])
print(match["field_readiness_status"], match["field_readiness_reasons"])
print(match["field_coverage_ratio"], match["field_missing_distinct_values"])
print(match["field_resolver_coverage_ratio"], match["field_resolver_missing_distinct_values"])
print(match["ambiguous"], match["ambiguity_kind"], match["ambiguity_group_size"])
print(catalog.resolve_anywhere("opponent Celtics", top_n=5)[0]["path"])
print(catalog.resolve_anywhere("Celtics", preferred_table="games", preferred_field="opponent_name"))
country = catalog.resolve("Germany", table="orders", field="ship_country", top_n=1)[0]
print(catalog.resolve_anywhere("apple", context_values=[country], top_n=5))
print(catalog.resolve_anywhere("curry", priors=[{"value": "Stephen Curry", "score": 1.0}]))
response = catalog.resolve_anywhere_response("customers from deutschland", top_n=5)
print(response["query"], response["candidate_count"], response["ambiguous"])
print(response["ambiguity"]["kinds"] if response["ambiguity"] else [])
responses = catalog.resolve_anywhere_many_response(["ticker aapl", "customers from deutschland"])
print([response["candidate_count"] for response in responses])
print(catalog.resolve_many(["apple", "globex"], table="orders", field="vendor_name"))
print(catalog.resolve_anywhere_many(["ticker aapl", "customers from deutschland"]))

catalog.save("shop.jmcat")
loaded = jawntmap.DatasetCatalog.load("shop.jmcat")
assert loaded.resolve("apple", table="orders", field="vendor_name", top_n=1)
print(loaded.summary())
print(loaded.summary()["semantic_type_counts"])
print(loaded.summary()["known_resolver_coverage_ratio"])
print(loaded.summary()["known_resolver_coverage_ratio_by_semantic_type"])
print(loaded.summary()["resolver_backed_semantic_type_counts"])
print([table["name"] for table in loaded.tables()])
print([table["name"] for table in loaded.tables(resolvable=True, semantic_type="location", resolver_backed=True)])
print(loaded.table("orders")["resolvable_field_count"])
print(loaded.table("orders")["known_resolver_coverage_ratio"])
print(loaded.table("orders")["resolver_backed_semantic_type_counts"])
print(loaded.tables()[0]["resolver_backed_field_count"])
print([field["path"] for field in loaded.fields(resolvable=True, semantic_type="location", resolver_backed=True)])
print(loaded.field("orders", "vendor_name")["readiness_status"])
print(loaded.field("orders", "vendor_name")["resolver_value_count"])
print(loaded.field_values("orders", "vendor_name", include_aliases=True, limit=5))
print(loaded.field_values_response("orders", "vendor_name", limit=5, include_resolver_values=True)["resolver_values"])
print([field["name"] for field in loaded.table_fields("orders", resolvable=True)])

csv_catalog = jawntmap.index_csv(
    "vendors.csv",
    table_name="vendors",
    delimiter="auto",
    normalize_headers=True,
    normalize_table_name=True,
)
assert csv_catalog.resolve("acme", table="vendors", field="vendor_name", top_n=1)

headerless_catalog = jawntmap.index_csv(
    "vendors.csv",
    table_name="vendors",
    delimiter="auto",
    has_headers=False,
)
assert headerless_catalog.resolve("acme", table="vendors", field="column_2", top_n=1)

record_catalog = jawntmap.index_records(
    [
        {"vendor_name": "Acme Industrial", "vendor_code": "ACME"},
        {"vendor_name": "Globex Corporation", "vendor_code": "GLOB"},
    ],
    table_name="vendors",
    normalize_names=True,
)
assert record_catalog.resolve("ACME", table="vendors", field="vendor_name", top_n=1)
```

Build directly from records or CSV exports when wiring Jawntmap into another
repo:

```python
records = [
    {"value": "Philadelphia 76ers", "aliases": ["Sixers", "PHI"]},
    {"value": "New York Knicks", "aliases": "Knicks|NYK"},
]

teams = EntityResolver.from_records(records, alias_separator="|")
assert teams.resolve("sixers", top_n=1)[0]["name"] == "Philadelphia 76ers"

companies = EntityResolver.from_csv(
    "companies.csv",
    value_column="company_name",
    aliases_column="aliases",
    delimiter="auto",
)

headerless_companies = EntityResolver.from_csv(
    "companies.csv",
    value_column="column_2",
    aliases_column="column_3",
    delimiter="auto",
    has_headers=False,
)
```

If records or exports include popularity/frequency signals, let the loader
turn them into conservative default tie-break priors:

```python
vendors = EntityResolver.from_records(
    [
        {"value": "Acme LLC", "aliases": ["Acme"], "row_count": 2},
        {"value": "Acme Ltd", "aliases": ["Acme"], "row_count": 100},
    ],
    frequency_key="row_count",
)

assert vendors.resolve_response("Acme", top_n=2)["candidates"][0]["name"] == "Acme Ltd"
vendors.save("vendors.jawntmap")
assert EntityResolver.load("vendors.jawntmap").resolve("Acme", top_n=1)[0]["name"] == "Acme Ltd"
```

Resolve multiple mentions against the same field with one call:

```python
batch = resolver.resolve_many(["aapl", "microsft"], top_n=1)

assert [matches[0]["name"] for matches in batch] == [
    "Apple Inc.",
    "Microsoft Corporation",
]
```

Persist a built resolver when the canonical values are expensive to collect or
normalize. Saved resolvers include aliases and field-specific configuration:

```python
resolver.save("companies.jawntmap")
restored = EntityResolver.load("companies.jawntmap")

assert restored.resolve("aapl", top_n=1)[0]["name"] == "Apple Inc."
```

Use `resolve_response` when a caller needs abstention/ambiguity diagnostics,
all generated candidates, or scoring explanations:

```python
ambiguous = EntityResolver(["Acme LLC", "Acme Ltd"])
response = ambiguous.resolve_response("Acme", top_n=2, include_explanations=True)

assert response["ambiguous"]
assert response["ambiguity"]["runner_up_candidate"] == "Acme Ltd"
assert "features" in response["candidates"][0]
```

Use `resolve_many_response` for the same diagnostics across a batch:

```python
responses = resolver.resolve_many_response(
    ["aapl", "microsft"],
    top_n=1,
    include_explanations=True,
)

assert responses[0]["candidates"][0]["name"] == "Apple Inc."
assert "explanation" in responses[1]["candidates"][0]
```

Request-time priors can break true lexical ties without rebuilding the
resolver. Priors may be dictionaries or tuples:

```python
resolved = ambiguous.resolve_with_priors(
    "Acme",
    [{"value": "Acme Ltd", "score": 1.0}],
    top_n=2,
)

assert resolved["candidates"][0]["name"] == "Acme Ltd"
assert not resolved["ambiguous"]

batch_resolved = ambiguous.resolve_many_response(
    ["Acme"],
    priors=[{"value": "Acme Ltd", "score": 1.0}],
)
assert batch_resolved[0]["candidates"][0]["name"] == "Acme Ltd"
```

Tune matching behavior per field at construction time. For code-like fields,
use stricter candidate generation so near codes abstain instead of fuzzing to a
neighbor:

```python
strict_codes = EntityResolver(
    ["ACCT-004096", "ACCT-004097"],
    max_edit_distance=0,
    min_trigram_similarity=1.0,
)

assert strict_codes.resolve("ACCT-004096", top_n=1)[0]["name"] == "ACCT-004096"
assert strict_codes.resolve("ACCT-004095", top_n=1) == []
assert strict_codes.config()["query"]["max_edit_distance"] == 0
```

For broad alias/name fields, keep the defaults or loosen
`max_edit_distance`, `min_trigram_similarity`, and `max_candidates`. For
precision-sensitive fields, raise `min_raw_score` or keep abstention enabled.

The wheel ships a `jawntmap.pyi` stub so editors and type checkers understand
the public resolver methods, response dictionaries, and prior formats.

For full schema-aware NL2SQL demos, benchmarks, and the Rust CLI, see the
GitHub repository.

