Metadata-Version: 2.4
Name: mongo-x-ray
Version: 1.5.1
Summary: MongoDB diagnostics toolkit with health checks and log analysis
Author: Yaoxing Zhang
License: """
        Copyright (c) 2025 MongoDB Inc.
        
        DISCLAIMER: THESE CODE SAMPLES ARE PROVIDED FOR EDUCATIONAL AND ILLUSTRATIVE PURPOSES ONLY,
        TO DEMONSTRATE THE FUNCTIONALITY OF SPECIFIC MONGODB FEATURES.
        THEY ARE NOT PRODUCTION-READY AND MAY LACK THE SECURITY HARDENING, ERROR HANDLING, AND TESTING REQUIRED FOR A LIVE ENVIRONMENT.
        YOU ARE RESPONSIBLE FOR TESTING, VALIDATING, AND SECURING THIS CODE WITHIN YOUR OWN ENVIRONMENT BEFORE IMPLEMENTATION.
        THIS MATERIAL IS PROVIDED "AS IS" WITHOUT WARRANTY OR LIABILITY.
        """
Project-URL: Homepage, https://github.com/mongodb-ps/ce-mongo-x-ray
Project-URL: Repository, https://github.com/mongodb-ps/ce-mongo-x-ray
Project-URL: Issues, https://github.com/mongodb-ps/ce-mongo-x-ray/issues
Keywords: mongodb,diagnostics,healthcheck,log-analysis
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Database
Classifier: Topic :: System :: Monitoring
Requires-Python: <4,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pymongo==4.17.0
Requires-Dist: Jinja2==3.1.6
Requires-Dist: Markdown==3.10.3
Requires-Dist: types-Markdown==3.10.2.20260712
Requires-Dist: MarkupSafe==3.0.3
Requires-Dist: packaging==26.2
Requires-Dist: Pygments==2.20.0
Requires-Dist: PyYAML==6.0.3
Requires-Dist: requests==2.34.2
Requires-Dist: urllib3==2.7.0
Requires-Dist: openai==2.51.0
Requires-Dist: chromadb==1.5.9
Requires-Dist: httpx==0.28.1
Requires-Dist: httpcore==1.0.9
Requires-Dist: h11==0.16.0
Requires-Dist: certifi==2026.7.22
Requires-Dist: idna==3.18
Requires-Dist: sniffio==1.3.1
Requires-Dist: anyio==4.14.2
Requires-Dist: python-dateutil==2.9.0.post0
Requires-Dist: types-python-dateutil==2.9.0.20260716
Requires-Dist: pymongoftdc>=0.1.1
Requires-Dist: python-dotenv==1.2.2
Requires-Dist: WeasyPrint==69.0
Provides-Extra: dev
Requires-Dist: ruff==0.16.1; extra == "dev"
Requires-Dist: pyright==1.1.411; extra == "dev"
Requires-Dist: pytest==9.1.1; extra == "dev"
Requires-Dist: pytest-cov==7.1.0; extra == "dev"
Requires-Dist: pluggy==1.6.0; extra == "dev"
Requires-Dist: iniconfig==2.3.0; extra == "dev"
Requires-Dist: tomli==2.4.1; extra == "dev"
Requires-Dist: pyinstaller==6.21.0; extra == "dev"
Requires-Dist: pyinstaller-hooks-contrib>=2026.6; extra == "dev"
Requires-Dist: macholib==1.16.4; extra == "dev"
Requires-Dist: altgraph==0.17.5; extra == "dev"
Dynamic: license-file

# x-ray
[![Makefile](https://github.com/mongodb-ps/ce-mongo-x-ray/actions/workflows/makefile.yml/badge.svg)](https://github.com/mongodb-ps/ce-mongo-x-ray/actions/workflows/makefile.yml)
[![Release](https://github.com/mongodb-ps/ce-mongo-x-ray/actions/workflows/release.yml/badge.svg)](https://github.com/mongodb-ps/ce-mongo-x-ray/actions/workflows/release.yml)
[![PyPI](https://img.shields.io/pypi/v/mongo-x-ray.svg)](https://pypi.org/project/mongo-x-ray/)


This project aims to create tools for MongoDB analysis and diagnosis. So far 3 modules are being built:
- Health check module.
- Log analysis module.
- `getMongoData` visualization module (Under construction).

## 1 Compatibility Matrix
### 1.1 Health Check
|  Replica Set  | Sharded Cluster | Standalone |
| :-----------: | :-------------: | :--------: |
| >=4.2 &check; |  >=4.2 &check;  |  &cross;   |

Older versions are not tested.

### 1.2 Log Analysis
Log analysis requires JSON format logs, which is supported since 4.4.
|  Replica Set  | Sharded Cluster |  Standalone   |
| :-----------: | :-------------: | :-----------: |
| >=4.4 &check; |  >=4.4 &check;  | >=4.4 &check; |


### 1.3 getMongoData Analysis
Analyze & visualize the [getMongoData.js](https://github.com/mongodb/support-tools/tree/master/getMongoData) output.
|  Replica Set  | Sharded Cluster | Standalone |
| :-----------: | :-------------: | :--------: |
| >=4.4 &check; |  >=4.4 &check;  |  &cross;   |

### 1.4 FTDC Analysis
Run a basic FTDC analysis.
|  Replica Set  | Sharded Cluster |  Standalone   |
| :-----------: | :-------------: | :-----------: |
| >=4.4 &check; |  >=4.4 &check;  | >=4.4 &check; |

## 2 How to Install
### 2.1 PyPi
#### 2.1.1 Install with Pip
The easiest and recommended way to install x-ray is to use `pip`:
```bash
pip install mongo-x-ray
```

#### 2.1.2 Build from Source
```bash
git clone https://github.com/mongodb-ps/ce-mongo-x-ray
cd x-ray
pip install .
```

### 2.2 PyInstaller
#### 2.2.1 Prebuilt Binaries
Currently the prebuilt binaries are available on 3 platforms:
- Ubuntu 22.04 (AMD64)
- MacOS 14 (ARM64)
- Windows 2022 (AMD64)

Download them from [Releases](https://github.com/mongodb-ps/ce-mongo-x-ray/releases).

#### 2.2.2 Build from Source
x-ray is tested on `Python 3.9.22`. On MacOS or Linux distributions, you can use the `make` command to build the binary:
```bash
git clone https://github.com/mongodb-ps/ce-mongo-x-ray
cd x-ray
make deps # if it's the first time you build the project
make # equal to `make build`
```

There are other make targets. Use `make help` to find out.

For Windows users, if `make` command is not available. You can use Python commands to build the binary:
```powershell
python.exe -m venv .venv
.venv\Scripts\python.exe -m pip install --upgrade pip
.venv\Scripts\python.exe -m pip install -e ".[dev]"
.venv\Scripts\python.exe -m PyInstaller --onefile `
  --name x-ray `
  --add-data="templates;templates" `
  --add-data="libs;libs" `
  --icon="misc/x-ray.ico" `
  --hidden-import=openai `
  x-ray
```

#### 2.3 For Developers
For developers, use `make deps` to prepare venv and dependencies
```bash
make deps
```
Or
```bash
python3 -m venv .venv
python3 -m pip install --upgrade pip
python3 -m pip install -e ".[dev]"
```

## 3 Using the Tool
```bash
x-ray [-h] [-q] [-c CONFIG] {healthcheck,hc,log,gmd,ftdc}
```
| Argument         | Description                                                                                                                                                                                         |        Default         |
| ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------: |
| `-q`, `--quiet`  | Quiet mode.                                                                                                                                                                                         |        `false`         |
| `-h`, `--help`   | Show the help message and exit.                                                                                                                                                                     |          n/a           |
| `-c`, `--config` | Path to configuration file.                                                                                                                                                                         | Built-in `config.json` |
| `command`        | Command to run. Include:<br/>- `healthcheck` or `hc`: Health check.<br/>- `log`: Log analysis.<br/>- `gmd`: getMongoData analysis.<br/>- `ftdc`: FTDC analysis.<br/>- `ingest`: Ingest a risk register CSV.<br/>- `version`: Show version info. |          None          |

Besides, you can use environment variables to control some behaviors:
- `ENV=development` For developing. It will change the following behaviors:
  - Formatted the output JSON for for easier reading.
  - The output will not create a new folder for each run but overwrite the same files.
- `LOG_LEVEL`: Can be `DEBUG`, `ERROR` or `INFO` (default).

### 3.1 Health Check Component
#### 3.1.1 Examples
```bash
./x-ray healthcheck localhost:27017 # Scan the cluster with default settings.
./x-ray hc localhost:27017 --output ./output/ # Specify output folder.
./x-ray hc localhost:27017 --config ./config.json # Use your own configuration.
```

#### 3.1.2 Full Arguments
```bash
x-ray healthcheck [-h] [-s CHECKSET] [-o OUTPUT] [-f {markdown,html,pdf}] [uri]
```
| Argument           | Description                                                                       |  Default  |
| ------------------ | --------------------------------------------------------------------------------- | :-------: |
| `-s`, `--checkset` | Checkset to run.                                                                  | `default` |
| `-o`, `--output`   | Output folder path.                                                               | `output/` |
| `-f`, `--format`   | Output format (`markdown`, `html`, or `pdf`). PDF also retains Markdown and HTML. |  `html`   |
| `uri`              | MongoDB database URI.                                                             |   None    |

For security reasons you may not want to include credentials in the command. There are 2 options:
- If the URI is not provided, user will be asked to input one.
- If URI is provided but not username/password, user will also be asked to input them.

#### 3.1.3 More Info
Refer to the wiki for more details.
- [Customize the thresholds](https://github.com/mongodb-ps/ce-mongo-x-ray/wiki/Health-Check-Configuration)
- [Database permissions](https://github.com/mongodb-ps/ce-mongo-x-ray/wiki/Health-Check-Database-Permissions)
- [Output](https://github.com/mongodb-ps/ce-mongo-x-ray/wiki/Health-Check-Output)
- [Customize the output](https://github.com/mongodb-ps/ce-mongo-x-ray/wiki/Health-Check-Output-Template)

### 3.2 Log Analysis Component
#### 3.2.1 Examples
```bash
# Full analysis
./x-ray log mongodb.log
# Time range filter
./x-ray log /var/log/mongodb/ 2026-07-20T08:00:00Z 2026-07-20T10:00:00Z
# For large logs, analyze a random 10% logs
./x-ray log -r 0.1 mongodb.log
# Discover log folders recursively
./x-ray log --discover /var/log/
```

#### 3.2.2 Full Arguments
```bash
x-ray log [-h] [-s CHECKSET] [-o OUTPUT] [-f {markdown,html,pdf}] [-r RATE] [--top TOP] [--discover] log_file [start_time] [end_time]
```
| Argument           | Description                                                                       |        Default         |
| ------------------ | --------------------------------------------------------------------------------- | :--------------------: |
| `-s`, `--checkset` | Checkset to run.                                                                  |       `default`        |
| `-o`, `--output`   | Output folder path.                                                               |       `output/`        |
| `-f`, `--format`   | Output format (`markdown`, `html`, or `pdf`). PDF also retains Markdown and HTML. |         `html`         |
| `-r`, `--rate`     | Sample rate. Only analyze a subset of logs.                                       |          `1`           |
| `--top`            | When analyzing the slow queries, only list top N.                                 |          `10`          |
| `--discover`       | Recursively search the given path for folders containing log files.               |        `false`         |
| `log_file`         | Path to the MongoDB log file or a folder of log files to analyze.                 |          n/a           |
| `start_time`       | Inclusive UTC start time in ISO-8601 format. Defaults to the first log line.      |          n/a           |
| `end_time`         | Inclusive UTC end time in ISO-8601 format. Defaults to the last log line.         |          n/a           |

### 3.3 getMongoData Analysis Component
#### 3.3.1 Examples
```bash
# getMongoData output for a sharded cluster
x-ray gmd misc/getMongoData-sh.json
# getMongoData output for a replica set
x-ray gmd misc/getMongoData-rs.json
```

#### 3.3.2 Full Arguments
```bash
x-ray gmd [-h] [-s CHECKSET] [-o OUTPUT] [-f {markdown,html,pdf}] gmd_file
```
| Argument           | Description                                                                       |  Default  |
| ------------------ | --------------------------------------------------------------------------------- | :-------: |
| `-s`, `--checkset` | Checkset to run.                                                                  | `default` |
| `-o`, `--output`   | Output folder path.                                                               | `output/` |
| `-f`, `--format`   | Output format (`markdown`, `html`, or `pdf`). PDF also retains Markdown and HTML. |  `html`   |

### 3.4 FTDC Analysis Component

The FTDC baseline analysis reports its capture timespan and effective sample rate, then
groups metrics into Workload, Read/Write Operations and Latencies, and
Performance sections. It includes operation rates and latencies, host memory
and CPU utilization, WiredTiger cache utilization, queue depth for each block
device, and free-space and utilization charts for every reported mount point.
Each metric shows its peak, average, unit, and a chart saved under the report
output's `charts` directory.
Start and end are inclusive UTC ISO-8601 timestamps. When omitted, the first
and last data points in the archive are used.

```bash
x-ray ftdc /var/lib/mongo/diagnostic.data
x-ray ftdc /var/lib/mongo/diagnostic.data 2026-06-17T08:00:00Z 2026-06-17T10:00:00Z
# Discover FTDC folders recursively
x-ray ftdc --discover /data/
```

```bash
x-ray ftdc [-h] [-s CHECKSET] [-o OUTPUT] [-f {markdown,html,pdf}] [-r RATE] [--svg] [--discover] ftdc_path [start_time] [end_time]
```
| Argument           | Description                                                          |        Default         |
| ------------------ | -------------------------------------------------------------------- | :--------------------: |
| `-s`, `--checkset` | Checkset to run.                                                     |       `default`        |
| `-o`, `--output`   | Output folder path.                                                  |       `output/`        |
| `-r`, `--rate`     | Controls FTDC sampling and accepts a value between `0` and `1`.      |  `1 / ingested files`  |
| `-f`, `--format`   | Output format (`markdown`, `html` or `pdf`). PDF also retains HTML.  |         `html`         |
| `--svg`            | Reference SVG charts instead of converting to PNG.                   |        `false`         |
| `--discover`       | Recursively search the given path for folders containing FTDC files. |        `false`         |
| `ftdc_path`        | Path to a directory containing FTDC files.                           |          n/a           |
| `start_time`       | FTDC time filter start.                                              | beginning of all files |
| `end_time`         | FTDC time filter end.                                                |    end of all files    |


```json
"BaselineAnalysisItem": {
  "chart_width": 450,
  "chart_height": 150
}
```

The fallback dimensions are defined in `ftdc_analysis/charts.py`.
Vertical grid lines are spaced every 100 pixels and horizontal grid lines every 50 pixels.
Workload and operation/latency charts use lines. Performance charts use bars.
Member-state charts are always 450×50 pixel bars.

#### 3.4.1 AI Analysis (Optional)
FTDC reports can include AI-generated summaries for each section (Workload,
Ops and Latencies, Performance). The analysis appears as a brief 2-3 sentence
assessment at the end of each section, flagging potential issues or confirming
normal operation.

**Configuration** — set the following environment variables:

| Variable            | Required | Default                  | Description                              |
| ------------------- | :------: | ------------------------ | ---------------------------------------- |
| `OPENAI_API_KEY`    |   Yes    | —                        | API key for the AI service               |
| `OPENAI_BASE_URL`   |    No    | OpenAI default           | Compatible API endpoint (e.g. DeepSeek)  |
| `AI_MODEL`          |    No    | `gpt-4o`                 | Model name to use                        |

If `OPENAI_API_KEY` is not set, AI analysis is silently skipped.

**Example** `.env` file:
```bash
OPENAI_API_KEY="sk-..."
OPENAI_BASE_URL="https://api.deepseek.com"
AI_MODEL="deepseek-v4-pro"
```

Or export directly in the shell:
```bash
export OPENAI_API_KEY="sk-..."
x-ray ftdc /var/lib/mongo/diagnostic.data
```

### 3.5 Risk Register Ingestion

The `ingest` command loads a CSV risk register into a local [ChromaDB](https://www.trychroma.com/)
database for later vector-search matching against issues found by the health check,
GMD, and log modules.

#### 3.5.1 Examples

```bash
x-ray ingest risks.csv            # Load risks from a CSV file
```

The CSV must have the following columns:

| Column           | Description                |
| ---------------- | -------------------------- |
| `ID`             | Unique risk identifier     |
| `Risk Level`     | Severity level (e.g. High) |
| `Impact`         | Business impact            |
| `Name`           | Short risk name            |
| `Risk Description` | Full description         |

Risks with the same `ID` are updated (upserted) on re-ingestion.
The database is stored at `~/.x-ray/chroma/`.

When a health check, GMD, or log report is generated, each found issue is
automatically matched against the risk register. Matching risks appear as a
blue `RISK-{ID}` badge in the Category column, with a hover tooltip showing
the full risk name and description. Only matches with &gt;50% similarity are
shown.

If the risk database is empty, matching is silently skipped with a yellow
warning in the logs.
