Metadata-Version: 2.4
Name: graphsearch-rag
Version: 0.5.0
Summary: GraphSearch: a GraphQL API server for Retrieval-Augmented Generation (RAG) over your documents
Author: GraphSearch contributors
License: MIT
Project-URL: Homepage, https://github.com/mohithgowdak/graphsearch
Project-URL: Issues, https://github.com/mohithgowdak/graphsearch/issues
Keywords: graphql,rag,semantic-search,llm,retrieval-augmented-generation
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Software Development :: Libraries :: Application Frameworks
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi>=0.110
Requires-Dist: strawberry-graphql[fastapi]>=0.220
Requires-Dist: uvicorn[standard]>=0.29
Requires-Dist: pydantic-settings>=2.2
Requires-Dist: numpy>=1.26
Requires-Dist: pypdf>=4.0
Requires-Dist: python-multipart>=0.0.9
Provides-Extra: local
Requires-Dist: sentence-transformers>=2.7; extra == "local"
Provides-Extra: openai
Requires-Dist: openai>=1.30; extra == "openai"
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.30; extra == "anthropic"
Provides-Extra: nltk
Requires-Dist: nltk>=3.8; extra == "nltk"
Provides-Extra: spacy
Requires-Dist: spacy>=3.7; extra == "spacy"
Provides-Extra: lda
Requires-Dist: scikit-learn>=1.3; extra == "lda"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: httpx>=0.27; extra == "dev"
Requires-Dist: ruff>=0.4; extra == "dev"
Requires-Dist: scikit-learn>=1.3; extra == "dev"
Dynamic: license-file

<div align="center">

<h1>⚡ GraphSearch</h1>

<p><strong>Ask questions. Get grounded answers. One GraphQL endpoint.</strong></p>

<p>Turn any pile of documents — PDFs, Markdown, plain text — into a typed,<br>
introspectable Q&amp;A API. Zero API keys, zero infrastructure to start.</p>

[![CI](https://github.com/mohithgowdak/graphsearch/actions/workflows/ci.yml/badge.svg)](https://github.com/mohithgowdak/graphsearch/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/graphsearch-rag?color=e10098&label=PyPI)](https://pypi.org/project/graphsearch-rag/)
[![Python](https://img.shields.io/pypi/pyversions/graphsearch-rag?color=blue)](https://pypi.org/project/graphsearch-rag/)
[![Docker](https://img.shields.io/badge/ghcr.io-graphsearch-2496ED?logo=docker&logoColor=white)](https://github.com/mohithgowdak/graphsearch/pkgs/container/graphsearch)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

<p>
<a href="#-quickstart">Quickstart</a> ·
<a href="#-the-playground">Playground</a> ·
<a href="#-graphql-api">API</a> ·
<a href="#-architecture">Architecture</a> ·
<a href="#-roadmap--good-first-issues">Roadmap</a> ·
<a href="#-contributing">Contributing</a>
</p>

<img src="docs/demo.gif" alt="GraphSearch demo: asking a question in GraphiQL and getting a grounded answer with ranked sources" width="850">

<p><em>"How do I get my money back?" finds the returns policy — no shared keywords, no API keys.</em></p>

</div>

---

## ✨ Why GraphSearch?

|  |  |
|---|---|
| 🔍 **Semantic search, zero keys** | Ships with offline embeddings — `pip install` and ask questions. Add sentence-transformers for real semantic retrieval, still 100% local. |
| 📎 **Grounded, cited answers** | Answers cite their sources as `[1]`, `[2]`, … mapping 1:1 to returned chunks with relevance scores. No hallucination hand-waving. |
| 📄 **PDFs welcome** | Drop PDF / Markdown / text files into the Playground, the `uploadFile` mutation, or the CLI — text extraction is built in. |
| 🧬 **One typed endpoint** | GraphQL means clients fetch exactly the fields they need, with introspection, GraphiQL, and a generated [TypeScript client](clients/typescript) that fails CI if it drifts from the schema. |
| 🔌 **Pluggable everything** | Embedder, vector store, and LLM are clean interfaces. Swap OpenAI ↔ Claude ↔ offline modes with one env var. |
| 🪶 **No infrastructure** | SQLite + numpy under the hood. No vector DB cluster, no queue, no Redis — until *you* decide you need one. |

## 🚀 Quickstart

```bash
pip install graphsearch-rag        # imports as `graphsearch`
```

```bash
graphsearch-ingest data/example_docs   # or your own .pdf / .md / .txt
graphsearch                            # → http://localhost:8000
```

That's it — no API keys, no services, no config. Prefer containers?

```bash
docker run -p 8000:8000 -v graphsearch-data:/data ghcr.io/mohithgowdak/graphsearch:latest
```

The image runs as a non-root user, ships a `HEALTHCHECK` against `/health`, and
persists its SQLite database in the `/data` volume. It bundles the `openai`,
`anthropic`, and `nltk` extras, so every chunking strategy except
`sentence_spacy` works offline out of the box. Invalid configuration (for
example a `CHUNK_OVERLAP` larger than `CHUNK_SIZE`, or an unknown chunking
strategy) fails at startup with an explicit message rather than on the first
upload.

> **`graphsearch` not recognized?** With `pip install --user` (Windows default
> when site-packages isn't writable), pip's Scripts folder may not be on PATH.
> Skip PATH entirely:
>
> ```bash
> python -m graphsearch                            # start the server
> python -m graphsearch.ingest data/example_docs   # ingest documents
> ```

## 🎮 The Playground

Open **http://localhost:8000/** and test RAG **with your own documents** before
writing a line of client code — paste text or drop files, ask questions, inspect
ranked sources. Every action has a **"Show the GraphQL"** toggle revealing the
exact query it runs, ready to copy into your app.

<div align="center">
<img src="docs/playground.png" alt="The GraphSearch Playground: your documents on the left, grounded answers with ranked sources on the right" width="850">
</div>

Prefer raw GraphQL? **GraphiQL** lives at http://localhost:8000/graphql.

## 🎚 Level up the pipeline

The default mode is fully offline (hashing-trick embeddings + extractive
answers) so the whole pipeline runs anywhere, including CI. Upgrade each stage
independently:

**Real semantic search — still no API key** (sentence-transformers, ~80 MB model, CPU):

```bash
pip install "graphsearch-rag[local]"
export GRAPHSEARCH_EMBEDDINGS=local    # $env:GRAPHSEARCH_EMBEDDINGS='local' on Windows
graphsearch-ingest data/example_docs   # re-ingest: embeddings are created at ingest time
graphsearch
```

**LLM-generated answers with citations** (OpenAI or Anthropic):

```bash
pip install "graphsearch-rag[anthropic]"   # or [openai]
export GRAPHSEARCH_LLM=anthropic           # or openai
export ANTHROPIC_API_KEY=sk-ant-...
graphsearch
```

| Setting | Options | Default |
|---|---|---|
| `GRAPHSEARCH_EMBEDDINGS` | `hash` (offline) · `local` (offline, semantic) · `openai` | `hash` |
| `GRAPHSEARCH_LLM` | `extractive` (offline) · `openai` · `anthropic` | `extractive` |

> Documents are embedded at ingest time — re-ingest after switching embedding backends.
> Full configuration reference: [.env.example](.env.example)

## 🧩 GraphQL API

```graphql
# Queries
answer(question: String!, topK: Int, expand: Int): Answer!   # RAG + optional neighbor expansion
search(query: String!, topK: Int, expand: Int): [Chunk!]!
documents(limit: Int = 20, offset: Int = 0): [Document!]!
document(id: ID!): Document

# Mutations
uploadDocument(content: String!, title: String, source: String, chunking: ChunkingStrategy): Document!
uploadFile(file: Upload!, title: String, source: String, chunking: ChunkingStrategy): Document!
deleteDocument(id: ID!): Boolean!
```

<details>
<summary><strong>Example: ingest and ask in one session</strong></summary>

```graphql
mutation {
  uploadDocument(
    content: "Support hours are 9am-5pm PST, Monday through Friday."
    title: "support-hours"
  ) { id chunkCount }
}

query {
  answer(question: "When is support available?") {
    text                                 # cites sources as [1], [2], ...
    sources { documentTitle text score }
  }
}
```

</details>

<details>
<summary><strong>File uploads (multipart)</strong></summary>

`uploadFile` follows the [GraphQL multipart request spec](https://github.com/jaydenseric/graphql-multipart-request-spec).
PDF text extraction happens server-side via pypdf — scanned/image-only PDFs are
rejected with a hint to OCR them first, encrypted PDFs with a hint to decrypt.

</details>

**TypeScript?** [`clients/typescript`](clients/typescript) ships a fully-typed
SDK generated straight from this schema — `client.Answer({ question })`,
`client.Search(...)`, etc. CI regenerates it on every push and fails the build
if it drifts from the server.

## 🏗 Architecture

```mermaid
flowchart LR
    C([Client]) -->|"GraphQL"| S["FastAPI + Strawberry"]
    S --> R["RagService"]
    R --> CH["Chunker<br/>paragraph · fixed · recursive · …"]
    R --> E["Embedder<br/>hash · local · OpenAI"]
    R --> V["VectorStore<br/>in-memory cosine"]
    R --> L["LLM<br/>extractive · OpenAI · Claude"]
    V <--> DB[("SQLite<br/>docs · chunks · vectors")]
    style S fill:#e10098,color:#fff
    style R fill:#1c1c28,color:#fff
```

Every pipeline stage is an abstract interface (`Chunker`, `Embedder`, `VectorStore`, `LLM`) —
new backends are drop-in additions, and several are up for grabs as
[good first issues](https://github.com/mohithgowdak/graphsearch/issues).

### Chunking strategies

Set `GRAPHSEARCH_CHUNKING` (or pass `chunking` on `uploadDocument` / `uploadFile`) to pick how documents are split. Default `paragraph` is the original packer. Also available: `fixed`, `sentence`, `recursive`, `markdown`, `html`, `latex`, `semantic` (uses the configured embedder), `sentence_nltk` / `sentence_spacy` / `topic_lda` (optional extras), and `contextual` (requires `GRAPHSEARCH_LLM=openai` or `anthropic`). Query-time neighbor expansion is `expand` on `search` / `answer` (or `GRAPHSEARCH_CHUNK_EXPAND`).

`topic_lda` needs `pip install graphsearch-rag[lda]` (scikit-learn). It treats each
sentence as a mini-document, fits Latent Dirichlet Allocation over the document,
and starts a new chunk wherever the dominant topic changes. Because the model is
fit per document rather than across a corpus, it needs a reasonably long text to
say anything useful — under six sentences it falls back to plain sentence
packing, and single-sentence topic flips are smoothed away so one noisy label
can't fragment a passage. Tune it with `GRAPHSEARCH_LDA_TOPICS` (default 5,
capped at half the sentence count) and `GRAPHSEARCH_LDA_MAX_ITER` (default 10).
The seed is fixed, so re-ingesting a document yields identical chunks. Expect it
to be the slowest strategy on large documents, since every ingest fits a model.

## 🛠 Development

```bash
git clone https://github.com/mohithgowdak/graphsearch && cd graphsearch
pip install -e ".[dev]"
ruff check .          # lint
pytest -v             # offline suite (hash embedder + extractive LLM)
```

## 🗺 Roadmap / good first issues

- [ ] Vector store backends: Qdrant, Weaviate, Redis/Valkey, pgvector, FAISS
- [ ] Embedding backends: Cohere, Voyage
- [ ] Streaming answers via GraphQL subscriptions
- [ ] Metadata tags + filters; hybrid keyword+vector search (SQLite FTS5)
- [ ] Auth (API keys / JWT) and rate limiting
- [ ] Query/embedding caching
- [x] Auto-generated TypeScript client — see [`clients/typescript`](clients/typescript)
- [ ] Evaluation harness + Prometheus metrics
- [ ] Advanced RAG: query rewriting, multi-hop retrieval, citation spans

Most of these are filed with implementation notes —
[grab one](https://github.com/mohithgowdak/graphsearch/issues) 👋

## 🤝 Contributing

Contributions are very welcome! [CONTRIBUTING.md](CONTRIBUTING.md) covers setup,
the backend-authoring guide, and PR guidelines. Look for the
[`good first issue`](https://github.com/mohithgowdak/graphsearch/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
label to get started.

## 📄 License

[MIT](LICENSE) — do whatever you want, just keep the notice.

---

<div align="center">
<sub>If GraphSearch saved you from wiring up a RAG pipeline by hand, consider giving it a ⭐ — it helps others find it.</sub>
</div>
