Metadata-Version: 2.4
Name: cve-dedup-checker
Version: 0.1.1
Summary: Parallel duplicate-check across free CVE and vulnerability databases for security researchers
Project-URL: Homepage, https://github.com/jashidsany/cve-dedup-checker
Project-URL: Repository, https://github.com/jashidsany/cve-dedup-checker
Project-URL: Issues, https://github.com/jashidsany/cve-dedup-checker/issues
Author-email: Jashid Sany <jashidsany@users.noreply.github.com>
License-Expression: MIT
License-File: LICENSE
Keywords: bug-bounty,cve,disclosure,duplicate-check,nvd,osv,patchstack,security,vulnerability,wpscan
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: httpx[http2]>=0.27.0
Requires-Dist: lxml>=5.2.0
Requires-Dist: platformdirs>=4.2.0
Requires-Dist: pydantic>=2.7.0
Requires-Dist: rich>=13.7.0
Requires-Dist: tomli>=2.0.0; python_version < '3.11'
Requires-Dist: typer>=0.12.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: respx>=0.21.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Description-Content-Type: text/markdown

# cve-dedup-checker

[![CI](https://github.com/jashidsany/cve-dedup-checker/actions/workflows/ci.yml/badge.svg)](https://github.com/jashidsany/cve-dedup-checker/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/cve-dedup-checker.svg?cacheSeconds=300)](https://pypi.org/project/cve-dedup-checker/)
[![Python](https://img.shields.io/pypi/pyversions/cve-dedup-checker.svg?cacheSeconds=300)](https://pypi.org/project/cve-dedup-checker/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Before you submit a CVE finding to WPScan, Patchstack, or HackerOne, check whether someone already reported it. `cve-dedup-checker` queries the major free vulnerability databases in parallel and tells you **clean**, **possible match**, or **confirmed duplicate**. Built for security researchers, bug bounty hunters, and disclosure coordinators. All data sources are free; no API keys required.

## Databases it queries

| Database | Auth | Notes |
|---|---|---|
| NVD (NIST) | none | REST API 2.0. Keyword and CPE search. |
| OSV (Google) | none | Aggregator covering GHSA, PyPA, RustSec, Go, Maven, NuGet, crates.io, and more. |
| GitHub Security Advisories | optional | Public. `GITHUB_TOKEN` bumps rate limit from 60/hr to 5000/hr but is not required. |
| WPScan | none | Scrapes the public plugin/theme page. |
| Patchstack | none | Scrapes the public vulnerability database page. |
| CISA KEV | none | Known Exploited Vulnerabilities JSON feed. |
| Exploit-DB | none | CSV index mirrored on GitLab. |

All responses are cached locally for 6 hours by default so you can iterate on a finding without re-hammering the free APIs.

## Install

```bash
pipx install cve-dedup-checker
```

Or from the latest `main`:

```bash
pipx install git+https://github.com/jashidsany/cve-dedup-checker.git
```

`pipx` puts the `cve-dedup-checker` command on your PATH automatically. Standard `pip install` also works if you prefer managing your own venv.

Requires Python 3.10 or later.

## Use

```bash
$ cve-dedup-checker wp-plugin <slug> --class broken-access-control
```

Exit codes:

- `0` clean
- `1` possible match (review required)
- `2` confirmed duplicate
- `3` inconclusive (no sources responded - usually a network issue)

### Target types

```bash
cve-dedup-checker wp-plugin <slug>
cve-dedup-checker wp-theme <slug>
cve-dedup-checker npm <package>
cve-dedup-checker pypi <package>
cve-dedup-checker github <owner/repo>
cve-dedup-checker cpe <cpe-2.3-string>
cve-dedup-checker keyword "<free text>"
```

### Output formats

```bash
cve-dedup-checker wp-plugin <slug> --output human      # default
cve-dedup-checker wp-plugin <slug> --output json       # pipe to jq, save to file
cve-dedup-checker wp-plugin <slug> --output markdown   # paste into writeup
```

### Narrow by vulnerability class

Accepts short codes or natural language; the matcher normalizes both:

```
xss                        sqli                  csrf
ssrf                       rce                   lfi
broken-access-control      info-disclosure       file-upload
deserialization            open-redirect         auth-bypass
```

```bash
cve-dedup-checker npm lodash --class prototype-pollution
cve-dedup-checker pypi django --class sqli
```

### Pipeline use

```bash
# Only proceed if clean
if cve-dedup-checker wp-plugin my-plugin --output json > /dev/null; then
  echo "No prior disclosure. Safe to submit."
fi
```

### Watch mode

During an active audit, you can keep an eye on whether a new advisory lands for your target before you submit. `--watch` re-runs the check on a fixed interval and prints only what's changed since the last run.

```bash
cve-dedup-checker wp-plugin my-plugin --class broken-access-control --watch 30m
cve-dedup-checker npm some-package --watch 6h
cve-dedup-checker pypi my-lib --watch 1d
```

- Supported units: `s`, `m`, `h`, `d` (e.g. `300s`, `15m`, `2h`, `1d`).
- Minimum interval is 60s to stay polite to free upstream APIs.
- State is stored at `~/.local/state/cve-dedup-checker/watch/` (or the OS equivalent).
- First run establishes a baseline. Subsequent runs print only new or resolved match URLs.
- Ctrl+C exits cleanly.

## Optional config

The tool works out of the box with zero config. If you want a higher GitHub rate limit, create `~/.config/cve-dedup-checker/config.toml`:

```bash
cve-dedup-checker config-sample > ~/.config/cve-dedup-checker/config.toml
```

Then uncomment the `github_token` line. Environment variables `GITHUB_TOKEN` and `WPSCAN_TOKEN` also work and take precedence over file config.

## Cache management

```bash
cve-dedup-checker cache-clear
```

Cache lives at `~/.cache/cve-dedup-checker/`.

## How dedup classification works

For every advisory returned from any source, the matcher:

1. **Identifier check** - does the advisory text contain the target's slug, package name, or a permutation of it?
2. **Class check** (if `--class` is supplied) - does the advisory text mention the same vulnerability class, via a synonym dictionary (XSS, sanitizer bypass, stored XSS, etc. all fold together)?

- Both match: **confirmed**.
- Only identifier matches: **possible** (different vuln class, review).
- No match: **clean**.

This is deliberately conservative: the tool flags things you might have missed, and you make the final judgment. It will never silently drop a hit you should see.

## Limitations

- Fuzzy matching is keyword-based, not semantic. Review possible matches yourself.
- Some sources have rate limits on the free tier (NVD is the tightest); the local cache mitigates this for iterative work.
- This tool is a **signal**, not a **verdict**. If it says clean and you suspect a dupe, keep searching (vendor bug trackers, Wordfence Intelligence, security mailing lists, HackerOne hacktivity).

## License

MIT - see `LICENSE`.
