Metadata-Version: 2.4
Name: opencode-pine2pyne
Version: 0.1.1
Summary: Pine Script v6 to PyneCore Python transpiler
Author: rubycell
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/rubycell/pine2pyne
Project-URL: Repository, https://github.com/rubycell/pine2pyne
Keywords: pinescript,tradingview,transpiler,pynecore,backtesting
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Office/Business :: Financial :: Investment
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: runtime
Requires-Dist: opencode-pyneruntime; extra == "runtime"
Dynamic: license-file

# PyneCore - Pine Script to Python Transpiler

> **Backlog & known bugs:** [Projects board #10](https://github.com/users/rubycell/projects/10)
> — open work is in **Backlog** (prioritised P0/P1/P2), shipped work in **Done**.
> Confirmed unfixed transpiler bugs live there too; several are pinned by strict
> `xfail` tests (see [Running Tests](#running-tests)).

## Installation (New Computer)

### Fastest: the sandbox-setup skill

From the repo root, run the bundled setup script (idempotent — safe to re-run):

```bash
bash .claude/skills/sandbox-setup/setup.sh
```

It builds `.venv`, installs everything below, detects `PINE_CPU_COUNT` /
`PINE_MAX_WORKERS`, seeds a demo dataset, and runs a smoke test that ends in
`smoke test : PASS` / `READY`. In Claude Code you can also invoke it as
`/sandbox-setup`.

### Manual (equivalent steps)

```bash
# From the repo root:
python3 -m venv .venv                          # 1. virtualenv
.venv/bin/pip install -e .                      # 2. this transpiler (editable)
.venv/bin/pip install 'opencode-pyneruntime[cli]'  # 3. PyneCore runtime + `pyne` CLI
.venv/bin/pip install optuna numpy              # 4. optimizer deps (not pulled in by the runtime)
source .venv/bin/activate                       # 5. activate (Linux/macOS)
# .venv\Scripts\activate                        #    (Windows)
```

Verify the install:

```bash
.venv/bin/python -c "import pynecore, pine2pyne, optuna, numpy; print('all ok')"
```

**Note**: All commands below assume you are at the repo root with the venv activated.

## How to Convert Pine Script to Python

### Single File Conversion

```bash
python -m pine2pyne path/to/script.pine -o path/to/output.py
```

### Batch Conversion

```bash
python -m pine2pyne "sample/pinescript/*.pine" -o workdir/scripts/
```

## Running Tests

There are **three** independent test systems. They answer different questions,
so a green run of one says nothing about the others.

| System | Question it answers | Runtime |
|---|---|---|
| **1. Unit tests** (`tests/`, pytest) | Are the transpiler's internals correct? | ~5 s |
| **2. Sample suites** (`tools/test_all_*.py`) | Does every corpus sample transpile and run? | ~35 s |
| **3. Ground truth** (`tools/*_ground_truth*`) | Does it produce the *same numbers* as before? | minutes |

### 1. Unit tests — pytest

2,000+ tests over `pine2pyne/` (93% line / 86% branch coverage).

```bash
.venv/bin/python -m pytest tests/ --ignore=tests/test_screener_regression.py -q

# with coverage
.venv/bin/python -m pytest tests/ --ignore=tests/test_screener_regression.py \
    --cov=pine2pyne --cov-report=term-missing
```

> `--ignore` is required: `tests/test_screener_regression.py` imports a
> `screener` module from a **sibling repo** that is absent here, and fails at
> collection — which aborts the whole run. Tracked as a P0 card on
> [board #10](https://github.com/users/rubycell/projects/10).

Some tests are `@pytest.mark.xfail(strict=True)` and pin **confirmed unfixed
bugs**, each with a card on [board #10](https://github.com/users/rubycell/projects/10).
Strict means fixing a bug turns its marker into a failure until the marker is
removed, so nothing regresses silently in either direction.

The highest-priority one is **P0: codegen emits arithmetically wrong Python** —
`10 - (5 - 3)` transpiles to `10 - 5 - 3`, which evaluates to 2 rather than 8.
See [docs/test_plan.md](docs/test_plan.md) for the full bug inventory and the
measured coverage/mutation results.

### 2. Sample suites — end-to-end transpile + run

Transpiles every `.pine` in the corpus and runs it through `pyne`.

```bash
python tools/test_all_samples.py          # sample/pinescript  (~411 files)
python tools/test_all_sample_pds.py       # sample_pds         (~328 files)

python tools/test_all_samples.py "ex_001*"      # filter by pattern
python tools/test_all_samples.py --timeout 30   # default 15s
python tools/test_all_samples.py --verbose      # stderr on failure
python tools/test_all_samples.py -j 0           # all CPU cores
```

- Source: `sample/pinescript/`, `sample_pds/` · Output: `workdir/scripts/`
- Data: `workdir/data/demo.ohlcv` · Results: `test_results{,_pds}.{json,txt}`

**Per-sample expectations** live in `tools/sample_expectations.toml`:

```toml
[expds_156_Out_of_bounds_index]
expect_error = "index_error"                    # MUST raise; running clean = FAIL

[ex_300_ticker_modify]
requires_feeds = { D = "demo", "2D" = "demo_2D" }   # -> --security D=demo …
```

`expect_error` marks Pine documentation samples that exist to *demonstrate* an
error — they are reported as `XFAIL` and count as passes. `requires_feeds`
supplies extra OHLCV feeds for samples calling `request.security()` at a
timeframe other than the chart's; pynecore deliberately will not resample the
chart feed, so the feed must be explicit. Generate one with:

```bash
cd workdir && pyne data aggregate data/demo.ohlcv -tf 2D
```

Shared logic for both suites lives in `tools/_suite_common.py` — edit it there,
not in each harness.

### 3. Ground truth — numeric regression

Diffs freshly generated output CSVs against ~1,985 stored baselines in
`ground_truth/` (779 MB). This is the only system that catches *silently wrong
numbers* — output that runs fine but computes something different.

```bash
python tools/generate_ground_truth.py     # (re)build baselines
python tools/compare_ground_truth.py      # diff current output vs baselines
python tools/compare_ground_truth.py --suite sample_pds -j 12
./tools/verify_ground_truth.sh            # end-to-end wrapper
```

> Regenerate baselines only when you have *confirmed* the new output is correct
> — otherwise a bug gets frozen in as the expected result.

### Test data

`workdir/data/` is gitignored, so a fresh clone has none and the sample suites
abort with `Data file not found`. Restore the exact dataset the recorded results
were measured against:

```bash
git show 227f3d2^:workdir/data/demo.ohlcv > workdir/data/demo.ohlcv
git show 227f3d2^:workdir/data/demo.toml  > workdir/data/demo.toml
```

(CCXT BTCUSDT 1D. `/sandbox-setup` instead seeds a small *synthetic* fixture —
fine for a smoke test, but results will not match the recorded rates.)

## For Claude Code / LLM Development

See `CLAUDE.md` in this directory for transpiler architecture, output format specs, optimizer documentation, and CSV rounding rules. See `pine2pyne/README.md` for transpiler internals and transformation rules.

## Optimizing Strategies

Quick start: `pyne optimize script.py data.ohlcv params.json -n 20`

See [Optimizer.md](./Optimizer.md) for full documentation (parameter JSON format, parallel execution, output files).

### Distributed Optimization (multi-machine)

Two tools distribute `pyne optimize` across SSH clusters:

| Tool | Model | Best for |
|------|-------|----------|
| `pyne-dynamic.sh` | Flat-queue (on-demand dispatch) | Multi-variant runs, heterogeneous clusters, long jobs |
| `pyne-parallel.sh` | Static pre-assignment | Quick jobs on similar-speed machines |

```bash
# Dynamic (recommended): flat-queue, pre-syncs once, auto-adapts to machine speed
# Run from workdir/ directory:
../tools/pyne-dynamic.sh scripts/strategy.py data/data.ohlcv optimize_variants/ \
  -H ../tools/machines.txt -C 24 --name my_run --output-dir runs/output/

# Static: pre-assigns chunks by core count
./tools/pyne-parallel.sh scripts/strategy.py data/data.ohlcv optimize.json \
  -H tools/machines.txt --sync

# Check progress / collect results
../tools/pyne-dynamic.sh --status
../tools/pyne-dynamic.sh --collect
```

Both use the same `machines.txt` format. See `CLAUDE.md` for cluster setup, machine file format, and troubleshooting.
