Metadata-Version: 2.5
Name: bedrock-probe
Version: 0.1.7
Summary: Test and search Amazon Bedrock model availability across regions, endpoints (runtime/mantle), APIs, and cross-Region inference scopes.
Project-URL: Homepage, https://github.com/jazst21/bedrock-probe
Project-URL: Repository, https://github.com/jazst21/bedrock-probe
Project-URL: Issues, https://github.com/jazst21/bedrock-probe/issues
Author: jazst21
License-Expression: MIT
License-File: LICENSE
Keywords: aws,bedrock,cli,converse,cross-region-inference,inference-profile,llm,model-availability
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Utilities
Requires-Python: >=3.9
Requires-Dist: boto3>=1.34.0
Requires-Dist: botocore>=1.34.0
Requires-Dist: rich-click>=1.8.0
Provides-Extra: dev
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: openai
Requires-Dist: aws-bedrock-token-generator>=1.0.0; extra == 'openai'
Requires-Dist: openai>=1.40.0; extra == 'openai'
Description-Content-Type: text/markdown

# bedrock-probe

[![PyPI](https://img.shields.io/pypi/v/bedrock-probe.svg)](https://pypi.org/project/bedrock-probe/)
[![Python](https://img.shields.io/pypi/pyversions/bedrock-probe.svg)](https://pypi.org/project/bedrock-probe/)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

**Test** and **search** Amazon Bedrock foundation models across regions,
endpoints, APIs, and cross-Region inference scopes — from one CLI.

`bedrock-probe` works against the two Bedrock inference surfaces:

- **`runtime`** — the AWS-native `bedrock-runtime` endpoint, invoked via the
  **Converse** API. Broadest coverage; automatically resolves cross-Region
  inference profiles (e.g. `us.openai.gpt-5.6-terra`) for models that require them.
- **`mantle`** — the OpenAI/Anthropic-native `bedrock-mantle` endpoint. Each model
  speaks exactly one of three API surfaces, and the tool auto-routes:
  - `chat`      → `/v1/chat/completions` (deepseek, qwen, gemma-3, gpt-oss, mistral, nvidia, minimax, zai, …)
  - `responses` → `/openai/v1/responses` (OpenAI GPT-5.x)
  - `messages`  → `/anthropic/v1/messages` (Anthropic Claude)

## Install

```bash
pip install bedrock-probe

# Optional: enable the mantle "--sdk openai" path (OpenAI SDK + Bedrock token)
pip install "bedrock-probe[openai]"
```

Requires Python 3.9+ and AWS credentials via the standard AWS CLI chain
(env vars, shared config, SSO cache, IAM role / IMDS). `boto3`/`botocore` and
`rich-click` are installed automatically.

## Usage

After install, the `bedrock-probe` command is available. `test` is the default
command, so you can omit it.

```bash
# Test every runtime model in one Region (Converse, default endpoint)
bedrock-probe --regions us-east-1

# Test the mantle endpoint (auto-routes chat / responses / messages)
bedrock-probe --regions us-east-1 --endpoint mantle

# Restrict by provider prefix (--labs) or id substring (--models)
bedrock-probe --regions us-east-1 --labs openai,anthropic
bedrock-probe --regions us-east-1 --models gpt-oss,deepseek

# Multiple Regions, more concurrency
bedrock-probe --regions us-east-1,us-west-2 --workers 8

# Also runnable as a module
python -m bedrock_probe --regions us-east-1
```

### `search` — availability discovery (no invocation)

Find **where** and **how** a model can be reached — region, endpoint, API, and
inference scope (in-region / geo / global) — plus the exact id to invoke with.

```bash
# Where is gpt-5.6-terra available, and how?
bedrock-probe search --models gpt-5.6-terra

# All Anthropic models, specific Regions, runtime only
bedrock-probe search --labs anthropic --regions us-east-1,us-west-2 --endpoint runtime
```

Example output:

```
 Model ID               Endpoint   Region      Scope       API         Invoke As
 ───────────────────────────────────────────────────────────────────────────────────────────
 openai.gpt-5.6-terra   mantle     us-east-1   in-region   responses   (bare)
 openai.gpt-5.6-terra   runtime    us-east-1   geo         converse    us.openai.gpt-5.6-terra
 openai.gpt-5.6-terra   runtime    eu-west-1   global      converse    global.openai.gpt-5.6-terra
```

`search` scans a built-in set of major commercial Regions when `--regions` is not
given, and scans **both** endpoints unless `--endpoint` restricts it.

## Options

| Option | Description |
|--------|-------------|
| `--regions` | AWS Region id(s), comma- or space-separated (also accepted positionally). |
| `--endpoint {runtime,mantle}` | Endpoint. `test` defaults to `runtime`; `search` scans both if unset. |
| `--labs` | Comma-separated provider prefixes (id before the first `.`), e.g. `openai,anthropic`. |
| `--models` | Comma-separated model-id substrings. Combines with `--labs`. |
| `--api` | `test` only. runtime: `converse` (default) / `converse_stream`; mantle: `auto` (default) / `chat` / `responses` / `messages`. |
| `--sdk` | (mantle only) `sigv4` (default, no extra deps) or `openai` (uses the OpenAI SDK; needs the `[openai]` extra + a Bedrock API key; covers chat + responses). |
| `--prompt` | The single prompt sent to every model. |
| `--max-tokens` | Max output tokens per request (default: 512). |
| `--workers` | Concurrent model calls per Region (default: 6). |
| `--timeout` | Per-model inference timeout in seconds (default: 45). |
| `--all-modalities` / `--text-only` | (runtime) also test non-text models. Default: text only. |
| `--stream` / `--no-stream` | Live per-model output (default on) vs a table at the end. |
| `--summary-table` | Also print the full per-Region table at the end. |
| `--profile` | AWS CLI profile (defaults to the standard credential chain). |
| `--output-dir` | Directory for the CSV output (default: current dir). |

## Output

- Live, colorized, streamed results (one line per model as it completes).
- A timestamped CSV report:
  - `test` → `<datetime>.bedrock.<endpoint>.test.csv`
  - `search` → `<datetime>.bedrock.search.csv`

Both the streamed output and the summary table include a **Scope** column
(`in-region` / `geo` / `global`) for every result. On `runtime` it reflects how
the model was reached (bare id vs. a cross-Region/`global.` inference profile);
on `mantle`, inference is always in-region, so it is shown as `in-region`.

Result statuses: **PASS** (answered), **EMPTY** (200 but no text), **FAIL**
(model/API error, e.g. wrong route or legacy model), **ERROR** (timeout / transport).

Press **Ctrl-C** at any time to stop; the tool exits promptly and writes partial
results.

## Notes

- Some models are only reachable via a specific API/scope. On `runtime`, models
  marked `INFERENCE_PROFILE`-only are auto-invoked through their cross-Region
  profile id (`us.`/`eu.`/`apac.`/`global.`). On `mantle`, `--api auto` tries
  chat → responses → messages until one accepts the model.
- The `mantle` endpoint is only officially available in a subset of Regions; a
  Region that answers `GET /v1/models` but is not officially supported may
  advertise models yet time out on inference. Such Regions are flagged.

## Development

```bash
git clone https://github.com/jazst21/bedrock-probe
cd bedrock-probe
pip install -e ".[dev,openai]"

python -m build          # build sdist + wheel into dist/
twine check dist/*       # validate metadata
```

## License

MIT — see [LICENSE](LICENSE).

> "Amazon Bedrock" and "AWS" are trademarks of Amazon.com, Inc. or its
> affiliates. This is an independent community project and is not affiliated
> with or endorsed by AWS.
