Metadata-Version: 2.4
Name: cortexdb-connectors
Version: 0.2.21
Summary: Data connectors for CortexDB — ingest business-system data into the v1 memory API.
Author-email: CortexDB Team <team@cortexdb.ai>
License-Expression: Apache-2.0
Project-URL: Homepage, https://cortexdb.ai
Project-URL: Documentation, https://cortexdb.ai/docs/connectors/slack
Project-URL: Support, https://cortexdb.ai/contact
Keywords: ai,memory,connectors,etl,slack,github,jira,cortexdb
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.0
Requires-Dist: croniter>=2.0
Provides-Extra: slack
Requires-Dist: slack-sdk>=3.0; extra == "slack"
Provides-Extra: github
Requires-Dist: pygithub>=2.0; extra == "github"
Provides-Extra: pagerduty
Requires-Dist: pdpyras>=5.0; extra == "pagerduty"
Provides-Extra: jira
Requires-Dist: jira>=3.0; extra == "jira"
Requires-Dist: tzdata; extra == "jira"
Provides-Extra: confluence
Requires-Dist: atlassian-python-api>=3.0; extra == "confluence"
Provides-Extra: notion
Requires-Dist: notion-client>=2.0; extra == "notion"
Provides-Extra: linear
Provides-Extra: discord
Requires-Dist: aiohttp>=3.9; extra == "discord"
Provides-Extra: teams
Requires-Dist: msal>=1.28; extra == "teams"
Provides-Extra: google-workspace
Requires-Dist: google-api-python-client>=2.0; extra == "google-workspace"
Requires-Dist: google-auth>=2.0; extra == "google-workspace"
Provides-Extra: salesforce
Requires-Dist: simple-salesforce>=1.12; extra == "salesforce"
Provides-Extra: hubspot
Requires-Dist: hubspot-api-client>=9.0; extra == "hubspot"
Provides-Extra: zendesk
Provides-Extra: intercom
Provides-Extra: servicenow
Provides-Extra: gitlab
Requires-Dist: python-gitlab>=4.0; extra == "gitlab"
Provides-Extra: webhooks
Requires-Dist: starlette>=0.37; extra == "webhooks"
Requires-Dist: uvicorn>=0.29; extra == "webhooks"
Requires-Dist: python-multipart>=0.0.9; extra == "webhooks"
Provides-Extra: all
Requires-Dist: slack-sdk>=3.0; extra == "all"
Requires-Dist: pygithub>=2.0; extra == "all"
Requires-Dist: pdpyras>=5.0; extra == "all"
Requires-Dist: jira>=3.0; extra == "all"
Requires-Dist: tzdata; extra == "all"
Requires-Dist: atlassian-python-api>=3.0; extra == "all"
Requires-Dist: notion-client>=2.0; extra == "all"
Requires-Dist: aiohttp>=3.9; extra == "all"
Requires-Dist: msal>=1.28; extra == "all"
Requires-Dist: google-api-python-client>=2.0; extra == "all"
Requires-Dist: google-auth>=2.0; extra == "all"
Requires-Dist: simple-salesforce>=1.12; extra == "all"
Requires-Dist: hubspot-api-client>=9.0; extra == "all"
Requires-Dist: python-gitlab>=4.0; extra == "all"
Requires-Dist: PyYAML>=6.0; extra == "all"
Requires-Dist: starlette>=0.37; extra == "all"
Requires-Dist: uvicorn>=0.29; extra == "all"
Requires-Dist: python-multipart>=0.0.9; extra == "all"
Dynamic: license-file

# cortexdb-connectors

Managed data connectors for [CortexDB](https://cortexdb.ai). Run them yourself, or let CortexDB host them for you on the [Starter tier](https://cortexdb.ai/pricing).

Each connector pulls from a third-party system (Slack, GitHub, Jira, …) and writes every event into CortexDB as a v1 experience.

## Plan availability (audit BLK-3)

| Mode | Plan | What it means |
|---|---|---|
| **Self-hosted** (`cortexdb-sync` CLI) | 🟢 **Free** (this package) | You run the poller / webhook receiver yourself. Cursor state lives in `~/.cortexdb/`. Every connector below works in this mode. |
| **Managed sync** (dashboard) | 🔒 **Starter+** ($29/mo) | CortexDB hosts the syncer, manages credentials, and surfaces health in the dashboard. The "Add Connector" button in the dashboard is gated on this plan. |

The free self-hosted path is always available; the managed dashboard path is what the pricing page calls "Starter+". The badge below appears on each connector's setup page for the same reason.

## Install

```bash
# core only — no third-party SDK dependencies
pip install cortexdb-connectors

# with one source's SDK
pip install 'cortexdb-connectors[slack]'

# everything
pip install 'cortexdb-connectors[all]'
```

Requires Python 3.10+.

## Auth

The connectors talk to the CortexDB v1 API. Two things you need:

- **A PASETO bearer token** (`Authorization: Bearer …`)
- **An actor id** that matches the token's `sub` claim (`X-Cortex-Actor: …`)

The fastest way to get both is to install [`cortexdb-cli`](https://pypi.org/project/cortexdb-cli/) and run `cortexdb init` — anonymous signup, no email or card, 7-day free-tier token. The connectors read `~/.cortexdb/state.json` automatically.

Or set env vars:

```bash
export CORTEXDB_URL=https://api-v1.cortexdb.ai
export CORTEXDB_API_KEY=v4.public...
export CORTEXDB_ACTOR=user:u_019e...
export CORTEXDB_TENANT_ID=acme                            # optional assertion; verified via whoami
export CORTEXDB_SCOPE_TEMPLATE='org:acme/source:{source}'   # optional
export CORTEXDB_SOURCE_INSTANCE_ID='acme-slack-workspace'   # see below
```

`{source}` (and `{scope}`, `{namespace}`, `{actor}`, `{entity.<type>}`) get filled in per event. (`{tenant}` is still accepted as a legacy alias of `{scope}`.)

`CORTEXDB_SOURCE_INSTANCE_ID` is an optional stable, non-secret account/site
identifier. Set it (or `--source-instance-id`) whenever two accounts of the
same provider write to the same CortexDB endpoint, actor, and scope. Its digest
partitions canonical source ids and the lifecycle ledger; never put an access
token or other credential in it. The empty default preserves existing
single-account identities. Connector-specific YAML may set
`source_instance_id` under that connector as well.

Before it opens durable lifecycle state, the connector resolves the billing
tenant from `GET /v1/auth/whoami`. `CORTEXDB_TENANT_ID` (or
`--cortex-tenant-id`) is an optional expected value and must match that
authenticated response; it never bypasses verification. Tenant identity is
part of the opaque ledger namespace, so credentials for a different tenant
cannot reuse completed source rows.

## Run

```bash
# one shot
cortexdb-sync sync slack

# poll loop, 60s between cycles
cortexdb-sync watch slack --interval 60

# resume from a specific point
cortexdb-sync sync github --since 2026-05-01T00:00:00Z

# see what's available
cortexdb-sync list

# check resolved CortexDB creds
cortexdb-sync auth

# cursor state across every connector
cortexdb-sync status
```

## Available connectors

All 18 connectors are **🟢 Free** to self-host and **🔒 Starter+** ($29/mo) to use via the managed dashboard.

```
slack            SLACK_BOT_TOKEN
github           GITHUB_TOKEN
gitlab           GITLAB_TOKEN
jira             JIRA_URL, JIRA_EMAIL, JIRA_API_TOKEN   (incl. crash-safe keyed attachments + multimodal extraction, 0.2.10+)
freshdesk        FRESHDESK_DOMAIN, FRESHDESK_API_KEY    (incl. crash-safe keyed attachments + multimodal extraction, 0.2.20+)
tldv             TLDV_API_KEY
linear           LINEAR_API_KEY
confluence       CONFLUENCE_URL, CONFLUENCE_EMAIL, CONFLUENCE_API_TOKEN
notion           NOTION_TOKEN
pagerduty        PAGERDUTY_API_KEY
discord          DISCORD_BOT_TOKEN
teams            TEAMS_TENANT_ID, TEAMS_CLIENT_ID, TEAMS_CLIENT_SECRET
google-workspace GW_SERVICE_ACCOUNT_KEY, GW_DELEGATED_USER
salesforce       SF_INSTANCE_URL, SF_CLIENT_ID, SF_CLIENT_SECRET, SF_USERNAME, SF_PASSWORD
hubspot          HUBSPOT_TOKEN
zendesk          ZENDESK_SUBDOMAIN, ZENDESK_EMAIL, ZENDESK_TOKEN
intercom         INTERCOM_TOKEN
servicenow       SNOW_INSTANCE, SNOW_USERNAME, SNOW_PASSWORD
```

Per-connector setup pages live at https://cortexdb.ai/docs/connectors/ (e.g. [Slack](https://cortexdb.ai/docs/connectors/slack)). **The "Add Connector" button on those pages is the managed (Starter+) path** — the CLI on this page is the free path.

For deep self-hosted operation — auth lifetimes, cursor state, retries, OAuth setup for Salesforce/Teams/Google Workspace — see [`SELF_HOSTING.md`](./SELF_HOSTING.md) in this directory.

## YAML config

If env vars get unwieldy, drop a `cortexdb-connectors.yaml` next to your sync invocation:

```yaml
slack:
  slack_bot_token: xoxb-...
  source_instance_id: acme-slack-workspace
  channels: [C01ABCDEF, C02GHIJKL]

github:
  github_token: ghp_...
  repos: [acme/api, acme/web]
  events: [pull_request, issue_comment]
```

## Programmatic use

Skip the CLI entirely:

```python
import asyncio
from cortexdb_connectors.slack import SlackConnector

connector = SlackConnector(
    cortex_url="https://api-v1.cortexdb.ai",
    cortex_api_key="v4.public...",
    scope="org:acme/source:slack",
    slack_bot_token="xoxb-...",
    channels=["C01ABC", "C02DEF"],
)
# actor + a richer scope_template are set after construction:
connector.bind(
    actor="user:u_019e...",
    scope_template="org:acme/source:slack/channel:{entity.channel}",
    source_instance_id="acme-slack-workspace",
    cortex_tenant_id="acme",  # optional assertion, verified with whoami
)

result = asyncio.run(connector.sync())
print(result.episodes_ingested, "ingested,", len(result.errors), "errors")
```

## Webhooks

The `cortexdb-connectors[webhooks]` extra ships a Starlette-based webhook receiver for sources that push (Slack Events API, GitHub Apps, Jira webhooks, …) instead of polling. See `cortexdb_connectors/webhooks.py`.

## Durable lifecycle ledger

Every canonical text or blob source is recorded in a local SQLite ledger before
a sync cursor advances or a webhook is acknowledged. It stores only opaque
source/version/idempotency digests, random upload tokens, and CortexDB
event/blob ids—never source text or file bytes. The default is
`~/.cortexdb/connector_event_ledger.sqlite3` (or under `CORTEXDB_STATE_DIR`);
override it with `CORTEX_CONNECTOR_EVENT_LEDGER_PATH`.
The authenticated billing tenant is included in its opaque namespace and an
optional configured tenant assertion must match `/v1/auth/whoami`.

Keep this database on durable local storage and back it up with SQLite's online
backup API (or stop writers and copy the database together with its `-wal` and
`-shm` files). A replica needs the same ledger to suppress replay beyond the
server idempotency window and to erase every historical source version, so run
one active connector writer per source namespace unless all writers share a
correctly coordinated ledger. Same-source generation changes use bounded,
crash-released OS advisory-lock stripes beside the SQLite file; network
filesystems are supported only when they provide equivalent locking semantics.
Accepted replacements retire older generations through exact derived-then-raw
erasure before the sync cursor advances; a failed cleanup remains durable for
retry. Rows are removed after confirmed source erasure or supersession;
monitor disk usage as part of connector operations.

New binary attachments use only CortexDB's versioned keyed-blob protocol. The
ledger commits a random, non-secret upload token before any blob network request,
then preflights and uploads the bytes through `/v1/blobs/idempotent`. A timeout
or restart reuses that same token and reconciles the authoritative blob ID, so
it cannot create a second physical object. Deletion tombstones the token through
the matching keyed endpoint before removing the local mapping. A missing,
disabled, malformed, or older server implementation is a hard sync failure;
connectors never fall back to an unkeyed upload. Existing blob-ID-only ledger
rows from releases before keyed uploads remain deletable through the legacy
ID route as a migration path, but no new rows are created that way.
Binary payloads over the 32 MiB source ceiling, malformed descriptors, and
invalid envelope metadata are rejected before whoami, ledger allocation, or
upload. Text-shaped attachments are strictly decoded and losslessly fragmented
at UTF-8 boundaries instead of taking the binary path.

## License

Apache-2.0
