Metadata-Version: 2.5
Name: jgtpricedb
Version: 0.1.2
Summary: Database-backed incremental price store for the JGT trading platform (PDSP heir)
Requires-Python: >=3.10
Requires-Dist: pandas>=1.5
Requires-Dist: sqlalchemy>=2.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Provides-Extra: postgres
Requires-Dist: psycopg[binary]>=3.1; extra == 'postgres'
Description-Content-Type: text/markdown

# jgt-pricedb

The persistence layer of the next-generation Price Service: bars live in a
database with an incremental, anchor-based refresh — CSV files become an
export, not the source of truth.

**Lineage**: implementation of the PDSP specifications reverse-engineered from
the Caishen .NET stack — `caishen/rispecs/PDSP/` (specs 70–77) — translated
into the jgt ecosystem's shapes.

## Structural Tension

- **Current reality**: price data as flat CSVs (`$JGTPY_DATA/pds/<INSTR>_<TF>.csv`);
  every refresh rewrites whole files and downstream recomputes whole series.
- **Desired state**: a database-backed store where refresh finds the anchor
  (the single forming bar per series) and writes only forward; downstream
  learns *which bar* changed and computes incrementally.

## Decisions

- **Periods are cut on the broker's trading session, not on UTC midnight**
  (0.1.1). The feed publishes H4, D1, W1 and M1 bars on a 17:00
  America/New_York boundary — 21:00Z in summer, 22:00Z in winter — so a UTC grid
  renamed 100% of them, silently. `m1`..`H1` stay on UTC, where they measurably
  already were. The session is stated per market, resolved inside `bar_key`, and
  recorded on the series; `tests/test_session_grid.py` is the measurement, run
  against the real holdings. See `rispecs/01-price-store.spec.md`,
  *The Session Grid*.
- **Portable schema** (SQLite for local-first dev, PostgreSQL for deployment);
  the anchor invariant is enforced by a partial unique index
  (`UNIQUE(series_id) WHERE is_forming`), not by application discipline.
- **No standalone importer.** Backfill is the refresh engine's bootstrap path
  (PDSP spec 72, Algorithm C) fed by a source adapter. `CsvSource` reads the
  existing `full/` CSVs through the exact same upsert path as `BrokerSource`
  live updates. The system can always rebuild itself from its own sources.
- **CSV compatibility export** (PDSP spec 75 pattern, 0.1.2): consumers
  (`jgtpy`, `jgtml`, `jgt-data-server`) keep reading the same file paths while
  the database becomes authoritative underneath. The format is measured rather
  than assumed — quotes keep the double the feed published, the derived columns
  are quantized at display precision +1 and +2 (with builtin `round`, which is
  not the one pandas reaches for), and the line terminator belongs to the file
  being replaced. A holdings file read in and written back out is byte-for-byte
  itself: **89 of 89** live files, 13 instruments, all seven timeframes, 95 208
  bars. See `rispecs/03-csv-export.spec.md`.
- **Redis pub/sub** (already in the jgt-data-server stack) carries
  bar-completed events in Phase 4 — the role Rebus/MSMQ played in Caishen.
- `pyproject.toml` with a **static version** — deliberately avoiding the
  `setup.py` circular-import pattern that blocks other jgt packages from
  installing in containers.

## Phases

| Phase | Creates | PDSP spec |
|---|---|---|
| 1 | Schema + store core + bootstrap-backfill through source adapters | 70, 71, 72 |
| 2 | Jobs (`jgtpdb`) replacing `refresh_data.sh` — shipped as `jgtpricedb-util` | 72, 74 |
| 3 | CSV export bridge — DB authoritative, downstream untouched (**0.1.2**) | 75 |
| 4 | Redis events → incremental IDS/CDS recompute | 73, 77 |
| 5 | Serving from DB; indicator tables; strategy layer (SpiderDb heir) | 76 |

## Layout

- `rispecs/` — jgt-native specifications (the buildable truth; start here)
- `src/jgtpricedb/` — the library
- `docker-compose.yml` — optional Postgres for deployment; SQLite needs nothing
- `../jgt-pricedb-util/` — the sibling package
  ([`jgtpricedb-util`](https://pypi.org/project/jgtpricedb-util/)): the jobs an
  operator runs against a store — bootstrap, refresh, a bounded forming-bar
  loop, an OANDA fetch on this session grid, and the freshness and relabel
  probes. This repository is the store; that one is the steward.
