Metadata-Version: 2.4
Name: curl-solve
Version: 0.1.0
Summary: Challenge-aware HTTP sessions on curl_cffi. Detect anti-bot challenges, persist cookies, and run pluggable solvers — a lightweight middle tier between impersonated HTTP and a real browser.
Author-email: Fawad Ali <fawadstar6@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Fawad Ali
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OR CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Repository, https://github.com/fawadss1/curl-solve
Project-URL: Issues, https://github.com/fawadss1/curl-solve/issues
Project-URL: Changelog, https://github.com/fawadss1/curl-solve/blob/master/CHANGELOG.md
Keywords: curl,curl_cffi,curl-impersonate,http,http2,http3,tls,tls fingerprint,browser impersonation,browser fingerprint,fingerprint,session,cookies,challenge,challenge solver,js challenge,anti-bot,anti-detection,bot detection,cloudflare,turnstile,akamai,datadome,web scraping,crawler,stealth,stealth http
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Security
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: curl_cffi>=0.16.1
Provides-Extra: dev
Requires-Dist: ruff>=0.15.13; extra == "dev"
Requires-Dist: mypy>=2.1.0; extra == "dev"
Requires-Dist: pytest>=9.0.3; extra == "dev"
Dynamic: license-file

# curl-solve

**Challenge-aware HTTP sessions on curl_cffi.**

`curl-solve` wraps [`curl_cffi`](https://github.com/lexiforest/curl_cffi) with
challenge detection, a cookie-aware retry loop, and pluggable solvers.

It is a **lightweight middle tier** — not a Cloudflare bypass:

```text
fast HTTP (curl_cffi)  →  curl-solve  →  real browser
```

## Comparison

| Feature                                 |   curl-solve   | curl_cffi |  cloudscraper  | requests |   Playwright   |
|-----------------------------------------|:--------------:|:---------:|:--------------:|:--------:|:--------------:|
| TLS / JA3 impersonation                 |       ✅       |    ✅     |       ❌       |    ❌    | ⚠️ real Chrome |
| HTTP/2                                  |       ✅       |    ✅     |       ❌       |    ❌    |       ✅       |
| HTTP/3 (QUIC)                           |       ✅       |    ✅     |       ❌       |    ❌    |       ✅       |
| Impersonate headers (`default_headers`) |       ✅       |    ✅     |   ⚠️ UA only   |    ❌    |      n/a       |
| Cookie session                          |       ✅       |    ✅     |       ✅       |    ✅    |       ✅       |
| Challenge detection (`kind`)            |       ✅       |    ❌     |       ❌       |    ❌    |       ❌       |
| Pluggable solvers                       |       ✅       |    ❌     |       ❌       |    ❌    |      n/a       |
| DNS overrides (`CURLOPT_RESOLVE`)       |       ✅       |    ✅     |       ❌       |    ❌    |       ⚠️       |
| Executes page JavaScript                |       ❌       |    ❌     | ⚠️ legacy IUAM |    ❌    |       ✅       |
| Cloudflare managed / Turnstile          |   ❌ browser   |    ❌     |       ❌       |    ❌    |       ✅       |
| Legacy CF IUAM (`jschl`)                | ⚠️ plugin hook |    ❌     |       ✅       |    ❌    |       ✅       |
| Scrapy-free standalone                  |       ✅       |    ✅     |       ✅       |    ✅    |       ✅       |
| Memory footprint                        |     🟢 Low     |  🟢 Low   |     🟢 Low     |  🟢 Low  |    🔴 High     |

> curl-solve sits **on** curl_cffi: same impersonated TLS, plus classification and a solver retry loop.
> **cloudscraper** can still clear *old* IUAM JS puzzles; it does not impersonate TLS and does not solve
> modern managed / Turnstile pages. When both return HTTP 200 product HTML, Cloudflare never issued
> that interstitial — impersonation (or a clean IP) passed, nothing “solved” the widget.

## Install

```bash
pip install curl-solve
```

Requires Python 3.11+ and `curl_cffi>=0.16.1`.

## Quick start

```python
from curl_solve import BROWSER_ONLY_KINDS, ChallengeSession

with ChallengeSession(impersonate="chrome") as session:
    resp = session.get("https://example.com")
    print(resp.status_code, resp.challenge.kind)
    if resp.challenge.kind in BROWSER_ONLY_KINDS:
        print("Escalate to a real browser.")
```

`ChallengeSession` keeps one impersonated TLS session. After each request it
classifies the response. If a registered solver can handle the challenge,
cookies are applied and the original request is retried.

## `ChallengeSession`

```python
from curl_solve import ChallengeSession

session = ChallengeSession(
    impersonate="chrome",
    proxy=None,
    timeout=30,
    http2=True,
    http3=False,
    dns_overrides=None,
    solvers=None,
    max_solves=1,
    raise_on_unsolved=False,
    default_headers=True,
    persist_cookies=True,
    # extra kwargs go to curl_cffi.requests.Session
)
```

| Argument            | Default                     | Description                                                                                                                                                       |
|---------------------|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `impersonate`       | `"chrome"`                  | curl_cffi browser target (TLS + JA3 + HTTP/2 fingerprint). Any `BrowserTypeLiteral` such as `"chrome"`, `"chrome150"`, `"firefox"`, `"safari"`.                   |
| `proxy`             | `None`                      | Session proxy URL (`http://`, `https://`, `socks4://`, `socks5://`). Applied to every request unless overridden.                                                  |
| `timeout`           | `30`                        | Default request timeout in seconds.                                                                                                                               |
| `http2`             | `True`                      | Use HTTP/2 (`CurlHttpVersion.V2_0`) when `http3` is false.                                                                                                        |
| `http3`             | `False`                     | Use HTTP/3 / QUIC (`CurlHttpVersion.V3`). Needs a UDP-capable path.                                                                                               |
| `dns_overrides`     | `None`                      | Host→IP map pinned via libcurl `CURLOPT_RESOLVE` (TLS SNI / Host stay the hostname). Example: `{"example.com": "203.0.113.10"}`.                                  |
| `solvers`           | `builtin_solvers()` (empty) | Plugin list. `None` uses builtins; `[]` disables plugins.                                                                                                         |
| `max_solves`        | `1`                         | Max solver attempts per request before giving up.                                                                                                                 |
| `raise_on_unsolved` | `False`                     | If `True`, raise `ChallengeUnsolved` when a challenge remains. If `False`, return the challenged response.                                                        |
| `default_headers`   | `True`                      | Let curl_cffi inject impersonate headers (User-Agent, `sec-ch-ua`, Accept, …).                                                                                    |
| `persist_cookies`     | `True`                      | `True`: send cookies on later requests. `False`: start each request with an empty jar; `session.cookies` then holds only this response's `Set-Cookie`. Per-call override: `persist_cookies=`. |
| `**session_kwargs`  | —                           | Forwarded to `curl_cffi.requests.Session` (for example extra `curl_options`).                                                                                     |

### Session properties

| Property                    | Description                                                                                                         |
|-----------------------------|---------------------------------------------------------------------------------------------------------------------|
| `session.http`              | Underlying `curl_cffi.requests.Session`. Extra fetches here **skip** the detect/solve loop (use this from solvers). |
| `session.cookies`           | Shared cookie jar (readable and writable).                                                                          |
| `session.headers`           | Default request headers on the inner session (readable and writable).                                               |
| `session.impersonate`       | Browser target used for this session.                                                                               |
| `session.proxy`             | Default proxy URL, or `None`.                                                                                       |
| `session.timeout`           | Default timeout in seconds.                                                                                         |
| `session.http_version`      | Resolved `CurlHttpVersion` (v1.1 / v2 / v3).                                                                        |
| `session.max_solves`        | Solver attempt limit.                                                                                               |
| `session.raise_on_unsolved` | Whether unsolved challenges raise.                                                                                  |
| `session.dns_overrides`     | Normalized host→IP map.                                                                                             |
| `session.solvers`           | Active `ChallengeSolver` list.                                                                                      |
| `session.persist_cookies`     | Whether this session persists cookies on later requests.                                                            |

### Lifecycle

| Method                                   | Description                                                                                 |
|------------------------------------------|---------------------------------------------------------------------------------------------|
| `session.close()`                        | Close the inner curl_cffi session and free connections.                                     |
| `with ChallengeSession(...) as session:` | Context manager; calls `close()` on exit.                                                   |
| `session.inspect(response)`              | Run `detect()` on any response-like object; returns a `Detection`. Does not send a request. |

---

## Request methods

All of these go through `session.request(...)`, so they share the same
detect → solver → retry loop, cookie jar, impersonate fingerprint, and
return type (`SolvedResponse`).

| Method                                   | Description                                                                                                               |
|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|
| `session.get(url, **kwargs)`             | HTTP GET. Fetch a document or API resource.                                                                               |
| `session.post(url, **kwargs)`            | HTTP POST. Submit a body (`data=`, `json=`, or `files=`).                                                                 |
| `session.put(url, **kwargs)`             | HTTP PUT. Replace a resource.                                                                                             |
| `session.patch(url, **kwargs)`           | HTTP PATCH. Partial update.                                                                                               |
| `session.delete(url, **kwargs)`          | HTTP DELETE. Remove a resource.                                                                                           |
| `session.head(url, **kwargs)`            | HTTP HEAD. Headers only; body is empty. Detection still runs on status/headers.                                           |
| `session.options(url, **kwargs)`         | HTTP OPTIONS. CORS / allowed-method probe.                                                                                |
| `session.request(method, url, **kwargs)` | Generic verb (`"GET"`, `"POST"`, `"PUT"`, `"PATCH"`, `"DELETE"`, `"HEAD"`, `"OPTIONS"`, or any string curl_cffi accepts). |

```python
resp = session.get(url)
resp = session.post(url, json={"q": "sku"})
resp = session.put(url, data=b"raw")
resp = session.patch(url, json={"stock": 1})
resp = session.delete(url)
resp = session.head(url)
resp = session.options(url)
resp = session.request("GET", url, solve=False)
```

### Shared request arguments

These work on **every** method above (`get` / `post` / … / `request`).

| Argument        | Description                                                                                                                          |
|-----------------|--------------------------------------------------------------------------------------------------------------------------------------|
| `url`           | Target URL (required).                                                                                                               |
| `solve`         | Default `True`. Run detect + solver + retry. `False` is a raw impersonated fetch (no solve loop).                                    |
| `proxy`         | Per-request proxy; overrides the session default for this call.                                                                      |
| `timeout`       | Per-request timeout in seconds.                                                                                                      |
| `http_version`  | Per-request HTTP version. Accepts `1` / `2` / `3`, `"v1.1"` / `"v2"` / `"v3"` / `"http2"` / `"http3"`, or a `CurlHttpVersion` value. |
| `persist_cookies` | Override session cookie persistence for this call only. `False` does not send the existing jar; this response's cookies are not kept. |

### Passed through to curl_cffi

Any other keyword is forwarded to `curl_cffi.requests.Session.request`:

| Argument          | Description                                                        |
|-------------------|--------------------------------------------------------------------|
| `params`          | Query string dict or list of pairs.                                |
| `data`            | Form fields or raw request body.                                   |
| `json`            | JSON body; sets `Content-Type: application/json`.                  |
| `headers`         | Extra headers for this call (merged with impersonate defaults).    |
| `cookies`         | Extra cookies for this call (merged into the jar for the request). |
| `files`           | Multipart file upload.                                             |
| `auth`            | HTTP basic/digest auth if curl_cffi supports it on this call.      |
| `allow_redirects` | Follow redirects (curl_cffi default applies if omitted).           |
| `verify`          | TLS certificate verification.                                      |
| `max_redirects`   | Redirect cap.                                                      |
| `**kwargs`        | Any other curl_cffi request option.                                |

---

## Response: `SolvedResponse`

Every request method returns a `SolvedResponse`. It wraps the curl_cffi
response and adds challenge metadata.

```python
print(resp.status_code)
print(resp.challenge.kind)
print(resp.text[:200])
```

### Challenge

`resp.challenge` is a `Detection` with a single field:

| Attribute             | Type            | Description                                         |
|-----------------------|-----------------|-----------------------------------------------------|
| `resp.challenge.kind` | `ChallengeKind` | Classified page type. `none` means a normal page.   |
| `resp.raw`            | curl_cffi       | Underlying response.                                |
| `resp.challenge`      | `Detection`     | Classification (always present).                    |

Escalate with `resp.challenge.kind in BROWSER_ONLY_KINDS`. A plugin may attempt
`resp.challenge.kind in SOLVABLE_KINDS` (currently only `cloudflare_iuam`).

### Body / HTTP fields (delegated)

Unknown attributes are forwarded to `resp.raw` (the curl_cffi response):

| Attribute          | Description                                |
|--------------------|--------------------------------------------|
| `resp.status_code` | HTTP status code.                          |
| `resp.url`         | Final URL after redirects.                 |
| `resp.text`        | Body decoded as text.                      |
| `resp.content`     | Body as `bytes`.                           |
| `resp.headers`     | Response headers.                          |
| `resp.cookies`     | Cookies set by this response.              |
| `resp.encoding`    | Declared or guessed encoding.              |
| `resp.ok`          | `True` if status is under 400 (curl_cffi). |
| `resp.elapsed`     | Transfer time, if curl_cffi provides it.   |
| `resp.json()`      | Parse body as JSON (curl_cffi method).     |

---

## `ChallengeKind`

| Kind                   | Value                  | Typical markers                                                          | Path                               |
|------------------------|------------------------|--------------------------------------------------------------------------|------------------------------------|
| `NONE`                 | `none`                 | Real content                                                             | done                               |
| `CLOUDFLARE_IUAM`      | `cloudflare_iuam`      | `jschl-answer`, `cf-browser-verification`                                | solvable plugin (none shipped yet) |
| `CLOUDFLARE_MANAGED`   | `cloudflare_managed`   | `just a moment`, `__cf_chl`, `challenges.cloudflare.com`, `cf-mitigated` | browser                            |
| `CLOUDFLARE_TURNSTILE` | `cloudflare_turnstile` | `cf-turnstile`, turnstile widget                                         | browser                            |
| `AKAMAI`               | `akamai`               | `sec-if-cpt-container`, `ak-challenge`                                   | browser                            |
| `DATADOME`             | `datadome`             | short-body `datadome` / `x-datadome` header                              | browser                            |
| `PERIMETERX`           | `perimeterx`           | `px-captcha`                                                             | browser                            |
| `GENERIC_JS`           | `generic_js`           | short-body `location.reload(true)`                                       | browser                            |
| `BLOCKED`              | `blocked`              | HTTP 403 / 429 / 503, or short-body block keywords                       | caller decides                     |

Large product pages that only mention a vendor name (for example `datadome`
in a long 200 body) are **not** treated as challenges.

How to read a result:

| `kind`                                                                                              | Meaning                                            |
|-----------------------------------------------------------------------------------------------------|----------------------------------------------------|
| `none`                                                                                              | Clean page.                                        |
| `cloudflare_iuam`                                                                                   | Plugin may try (`SOLVABLE_KINDS`). None ships yet. |
| `cloudflare_managed` / `cloudflare_turnstile` / `akamai` / `datadome` / `perimeterx` / `generic_js` | Escalate to browser (`BROWSER_ONLY_KINDS`).        |
| `blocked`                                                                                           | HTTP 403 / 429 / 503 or a short block page.        |

---

## Detection without a session

```python
from curl_solve import detect, ChallengeKind, is_js_challenge, is_blocked, is_session_ban

result = detect(status=403, body="<html>just a moment</html>")
assert result.kind is ChallengeKind.CLOUDFLARE_MANAGED
```

| Function                                                         | Description                                                                                                                                                                                  |
|------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `detect(response=None, *, status=None, headers=None, body=None)` | Classify a response object or raw `status` / `headers` / `body`. Accepts Scrapy-style (`status`, `body`) or requests-style (`status_code`, `text` / `content`) objects. Returns `Detection`. |
| `is_js_challenge(body)`                                          | `True` if the HTML looks like a JS challenge page (not a plain block keyword).                                                                                                               |
| `is_blocked(response=None, *, status=None, body=None)`           | `True` for HTTP 403 / 429 / 503 or anti-bot keywords (any body size).                                                                                                                        |
| `is_session_ban(response=None, *, status=None, body=None)`       | `True` when the HTTP session should be recycled: block status always; challenge/keyword heuristics only on short bodies.                                                                     |

---

## Pluggable solvers

```python
from curl_solve import ChallengeSession, ChallengeSolver, SOLVABLE_KINDS, SolveOutcome


class MySolver(ChallengeSolver):
    name = "my-solver"

    def can_handle(self, response, detection):
        return detection.kind in SOLVABLE_KINDS

    def solve(self, client, response, detection):
        # client.http.get(...) — extra fetches skip the solve loop
        return SolveOutcome.ok(cookies={"cf_clearance": "..."})
        # or: return SolveOutcome.failed("could not parse challenge")


session = ChallengeSession(solvers=[MySolver()])
```

| Piece                                                 | Description                                                                                     |
|-------------------------------------------------------|-------------------------------------------------------------------------------------------------|
| `ChallengeSolver.can_handle(response, detection)`     | Return `True` to take this challenge.                                                           |
| `ChallengeSolver.solve(client, response, detection)`  | Attempt a solve; return `SolveOutcome`.                                                         |
| `SolveOutcome.ok(cookies=..., retry=True, reason="")` | Success. Cookies are merged into the session; original request is retried when `retry` is true. |
| `SolveOutcome.failed(reason="", retry=False)`         | Failure. No retry unless you set `retry=True`.                                                  |
| `builtin_solvers()`                                   | Default plugin list. Phase 1 returns `[]`.                                                      |

Unsolved challenges are returned (or raised when `raise_on_unsolved=True`)
so a caller — later, scrapy-stealth `driver="auto"` — can escalate:
**turbo → curl-solve → browser**.

---

## Exceptions

| Exception                  | When                                                                                    |
|----------------------------|-----------------------------------------------------------------------------------------|
| `CurlSolveError`           | Base class.                                                                             |
| `ChallengeUnsolved`        | Challenge left unsolved and `raise_on_unsolved=True`. Has `.detection` and `.response`. |
| `CurlSolveTimeoutError`    | curl_cffi timed out. Also a `TimeoutError`.                                             |
| `CurlSolveConnectionError` | DNS, proxy, or connection failure. Also a `ConnectionError`.                            |
| `DependencyError`          | `curl_cffi` failed to import (often a missing VC++ runtime on Windows).                 |

---

## License

MIT
