Metadata-Version: 2.4
Name: dbt-pathfinder
Version: 0.2.0
Summary: CLI and library for exploring dbt manifests as graphs.
License-File: LICENSE
Requires-Python: >=3.9
Requires-Dist: networkx>=3.0
Requires-Dist: pydantic>=2.0
Requires-Dist: rich>=13.0
Requires-Dist: typer>=0.9
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Description-Content-Type: text/markdown

# dbt-pathfinder

> 🔎 Explore, debug, and understand your dbt lineage — fast.

`dbt-pathfinder` is a Python CLI + library that parses your `manifest.json` and turns it into a navigable graph of your dbt project.

It helps you answer questions like:

* “What breaks if I change this model?”
* “Which nodes and tests are affected by these changed files?”
* “How are these two models connected?”
* “What joins actually exist along this path?”
* “Is this relationship 1:1 or 1:N?”

---

## 🚀 Why this exists

dbt gives you lineage… but not *exploration*.

When projects scale:

* lineage graphs get noisy
* impact analysis becomes manual
* understanding joins requires digging through SQL

`dbt-pathfinder` bridges that gap:
👉 fast CLI exploration
👉 precise impact analysis
👉 inferred join + cardinality insight

---

## ✨ Features

### 🔍 `show`

Inspect a node and its immediate relationships

```bash
dbt-pathfinder show --manifest target/manifest.json --model fct_orders
```

Outputs:

* node metadata
* upstream dependencies
* downstream children

---

### 💥 `impact`

See everything downstream of a model (grouped by depth)

```bash
dbt-pathfinder impact --manifest target/manifest.json --model stg_orders
```

Example output:

```
Depth 1:
- fct_orders
- dim_customers

Depth 2:
- mart_customer_ltv
```

Perfect for:

* change impact analysis
* safe refactoring
* CI/CD validation

---

### 🎯 `ci-impact`

From a list of changed project files (paths as in `manifest` `original_file_path`), see downstream blast radius, impacted tests, and a suggested `dbt build --select` command. Built for CI/CD (for example, pass files touched in a PR).

```bash
dbt-pathfinder ci-impact \
  --manifest target/manifest.json \
  --files models/staging/stg_orders.sql \
  --files models/marts/dim_customers.sql
```

Example output (text):

```
Changed files:
- models/staging/stg_orders.sql
- models/marts/dim_customers.sql

Changed nodes:
- stg_orders
- dim_customers

Impacted downstream:
Depth 1:
- fct_orders
- mart_customer_ltv

Depth 2:
- customer_health_dashboard

Impacted tests:
- unique_fct_orders_order_id
- not_null_dim_customers_customer_id

Suggested command:
dbt build --select stg_orders+ dim_customers+
```

Options:

* `--output text` (default) or `--output json` (JSON uses node `unique_id` values in `impacted_by_depth`; text uses short labels)
* `--ui rich` (default) or `--ui text` when using `--output text` (same pattern as `impact` and `path-explain`; JSON always prints plain text)
* `--include-tests` (default) or `--no-include-tests` to omit the impacted-tests section and skip computing it

Repeat `--files` once per path. File matching is against each node’s `original_file_path` (`.sql` and `.yml`).

---

### 🧭 `path`

Find the shortest path between two nodes

```bash
dbt-pathfinder path \
  --manifest target/manifest.json \
  --from stg_orders \
  --to mart_customer_ltv
```

Supports:

* `--mode directed` (default)
* `--mode any` (for debugging unexpected relationships)

---

### 🧠 `path-explain` (🔥 killer feature)

Explain *how* models are connected — not just that they are.

```bash
dbt-pathfinder path-explain \
  --manifest target/manifest.json \
  --from stg_orders \
  --to mart_customer_ltv
```

Includes:

* inferred join conditions (`JOIN ... ON ...`)
* join keys
* inferred cardinality:

  * `1:1`
  * `1:N`
  * `N:1`
  * `N:N`
* confidence level

> ⚠️ Note: inference is heuristic and depends on SQL structure and dbt tests.

---

## 📦 Installation

### From PyPI (recommended)

```bash
pip install dbt-pathfinder
```

### From source (dev)

```bash
git clone https://github.com/<your-username>/dbt-pathfinder.git
cd dbt-pathfinder
pip install -e ".[dev]"
```

---

## ⚡ Quick Start

1. Generate your dbt manifest:

```bash
dbt compile
```

2. Run:

```bash
dbt-pathfinder show --manifest target/manifest.json --model my_model
```

---

## 🧰 CLI Usage

```bash
dbt-pathfinder --help
```

Examples:

```bash
dbt-pathfinder show --manifest target/manifest.json --model fct_orders --verbose

dbt-pathfinder impact --manifest target/manifest.json --model stg_orders

dbt-pathfinder impact --manifest target/manifest.json --model stg_orders --output json

dbt-pathfinder ci-impact \
  --manifest target/manifest.json \
  --files models/staging/stg_orders.sql \
  --files models/marts/dim_customers.sql

dbt-pathfinder ci-impact --manifest target/manifest.json --files models/staging/stg_orders.sql --output json

dbt-pathfinder path --manifest target/manifest.json --from stg_orders --to mart_customer_ltv

dbt-pathfinder path-explain --manifest target/manifest.json --from stg_orders --to mart_customer_ltv --ui rich
```

---

## 🖥 Output Modes

* `--ui rich` (default): formatted terminal output (`show`, `path`, `impact`, `path-explain`, `ci-impact` when using text output)
* `--ui text`: plain text
* `--output json`: machine-readable (great for CI/CD; supported on `impact`, `path-explain`, and `ci-impact`)

---

## 🧱 Library Usage

You can also use `dbt-pathfinder` programmatically:

```python
from dbt_pathfinder.services.pathfinder_service import PathfinderService
from dbt_pathfinder.services.ci_impact_service import CIImpactService

service = PathfinderService.from_manifest("target/manifest.json")

print(service.show("fct_orders"))
print(service.impact("stg_orders"))
print(service.path("stg_orders", "mart_customer_ltv", directed=True))
print(service.explain_path("stg_orders", "mart_customer_ltv", directed=True))

ci = CIImpactService.from_manifest("target/manifest.json")
result = ci.build_result(
    ["models/staging/stg_orders.sql"],
    include_tests=True,
)
print(result.model_dump())
```

---

## 🧠 How it works

* Parses dbt `manifest.json`
* Builds a directed graph:

  ```
  upstream → downstream
  ```
* Uses:

  * graph traversal for lineage + paths
  * SQL parsing heuristics for join inference
  * dbt tests (`unique`, `not_null`) for cardinality hints

---

## ⚠️ Limitations

* Join inference is best-effort (complex SQL/macros may reduce accuracy)
* Requires a valid dbt `manifest.json`
* Ambiguous model names may require full `unique_id`

---

## 🛠 Roadmap

* [x] CI impact from changed files (`ci-impact`)
* [ ] `ci-impact --git-diff` (range-based changed files)
* [ ] Visualization (graph output / web UI)
* [ ] dbt Cloud integration
* [ ] Column-level lineage support
* [ ] Better SQL parsing for complex joins

---

## 🤝 Contributing

Contributions welcome!

1. Fork the repo
2. Create a feature branch
3. Add tests
4. Open a PR

---

## ⭐ Support

If this tool helps you:

* ⭐ Star the repo
* 🐛 Open issues
* 💡 Suggest features

---

## 📄 License

MIT License
