Metadata-Version: 2.4
Name: addereq-dm
Version: 2.0.0
Summary: SDK for geophysics time-series data access, lightweight processing, and plotting
Author-email: WANG Qinglin <chd_wql@qq.com>
License: MIT
Project-URL: Homepage, https://github.com/chdwql/addereq-dm
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.25.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: gmssl>=3.2.0
Requires-Dist: tenacity>=8.0.0
Requires-Dist: pydantic>=2.0.0
Provides-Extra: viz
Requires-Dist: matplotlib>=3.3.0; extra == "viz"
Requires-Dist: numpy>=1.21.0; extra == "viz"
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.8; extra == "dev"
Requires-Dist: mypy>=0.910; extra == "dev"
Dynamic: license-file

# addereq-dm

Dameng-backed earthquake precursor time-series data access, processing, and plotting SDK. This package is the API-based successor to the Oracle-oriented `addereq` package.

## Install

Python 3.9 or newer is required.

The package is installed as `addereq-dm` and imported as `addereq_dm`:

```bash
pip install "addereq-dm[viz]"
```

```bash
pip install -e .
```

With plotting dependencies:

```bash
pip install -e ".[viz]"
```

## Connection config

For CLI usage, the recommended place for credentials is a user-level config file instead of the project directory.

Default config file location:
- Windows: `%APPDATA%\\addereq-dm\\config.env`
- Linux/macOS: `~/.config/addereq-dm/config.env`

Initialize the config from CLI with your own values:

```bash
addereq-dm init-config \
  --base-url http://your-api-server:8080 \
  --app-id your_app_id \
  --secret your_secret
```

If any required value is omitted, the CLI will prompt for it interactively. `timeout` defaults to `10`.

Or initialize from Python:

```python
from addereq_dm import initialize_user_config

initialize_user_config(
    base_url="http://your-api-server:8080",
    app_id="your_app_id",
    secret="your_secret",
)
```

Example written file:

```env
DM_API_BASE_URL="http://your-api-server:8080"
DM_API_APP_ID="your_app_id"
DM_API_SECRET="your_secret"
DM_API_TIMEOUT="10"
```

The CLI resolves values in this order:
- command-line arguments
- `--env-file` if provided
- the default user config file above
- current process environment variables
- built-in defaults such as `DM_API_TIMEOUT=10`

Business parameters such as station, point, item, sample rate, and time range are still expected to be passed explicitly in code or CLI commands.

## Quick start

```python
from addereq_dm import create_geophysics_client

api = create_geophysics_client(
    base_url="http://your-api-server:8080",
    app_id="your_app_id",
    secret="your_secret",
)
```

## Core fetch API

```python
df = api.ts.fetch_dys(
    station="taian_center",
    point="1",
    item="vertical_z",
    sample_rate="02",
    data_sample_rate="02",
    start_time="2025-11-01 00:00:00",
    end_time="2025-11-03 00:00:00",
)
```

`sampleRate` and `dataSampleRate` are not duplicates:
- `sampleRate`: raw input sample rate
- `dataSampleRate`: output sample rate for the returned series

Accepted sample-rate aliases currently include:
- `01`, `minute`, `min`
- `02`, `second`, `sec`
- `60`, `hour`, `h`
- `90`, `day`, `d`
- common Chinese aliases are also supported in code

Supported input styles:
- `station`: station id or station name
- `point`: point id, point number, or point name; may be omitted
- `item`: item id or item name

When resolution fails or the query is ambiguous, the resolver returns candidate suggestions to help narrow the scope.

Long-range requests are handled automatically. When the requested range spans more than 30 natural days, the SDK splits the request into multiple upstream calls, merges the returned frames, sorts by timestamp, and deduplicates boundary rows. Chunk boundaries follow an exclusive `end_time` rule, so each chunk covers `[start_time, end_time)` and the next chunk starts exactly at the previous chunk's `end_time`.

Use the next midnight when requesting a complete final day. For example, data for November 1-2 should use `end_time="2025-11-03 00:00:00"`.

## Scope keywords

The following keywords are supported for broad queries:
- `"all"`
- `"*"`
- common Chinese equivalents for “all” are also supported in code

## Lightweight processing

```python
stats = api.ts.summarize(df)
report = api.ts.quality_report(df)
hourly = api.ts.resample(df, "1h")
centered = api.ts.center(hourly)
detrended = api.ts.detrend(df, method="mean")
cleaned = api.ts.clip_outliers(df, zscore=3.0)
differenced = api.ts.difference(df)
robust = api.ts.hampel(df, window_size=15, threshold=5.0)
complete = api.ts.expand_timeline(df)
aligned = api.ts.align([df1, df2], labels=["station_a", "station_b"])
```

The processing layer normalizes sentinel missing values such as `999999` into real missing values before summaries, quality checks, resampling, alignment, and plotting.

Frames containing multiple logical series are processed independently by `STATIONID`, `POINTID`, and `ITEMID`. Multi-series summaries and quality reports include a `series` list with per-series results. Pass `group_cols=[]` only when an intentional whole-frame calculation is required.

Normalized output uses `SAMPLERATE` for the actual returned series rate and `SOURCE_SAMPLERATE` for the upstream source rate. `VALUE` is the canonical value column; `OBSVALUE` remains as a compatibility mirror for existing `addereq` workflows.

`expand_timeline` uses the sample rate to insert timestamps omitted by the API. `MISSING_REASON="missing_record"` identifies inserted rows, while `MISSING_REASON="sentinel"` identifies explicit upstream values such as `999999`.

## Optional metadata snapshot

Online resolution and plotting use live API metadata plus process-local memory caching. They do not automatically read or write persistent metadata. If offline plotting is required, explicitly create a station, point, item, and unit snapshot under the same user configuration directory as `config.env`.

Snapshot namespaces are SHA-256 hashes of the normalized API base URL; credentials are never written into snapshot files or keys. Refresh failures are reported directly and never hidden by silently using old metadata.

```bash
addereq-dm metadata path
addereq-dm metadata status
addereq-dm metadata refresh
addereq-dm metadata refresh --station-id 37001
addereq-dm metadata clear
```

## Optional batch fetch report

```python
df, report = api.ts.fetch_dys_with_report(
    station="all",
    point="1",
    item="3123",
    sample_rate="02",
    data_sample_rate="02",
    start_time="2025-11-01 00:00:00",
    end_time="2025-11-03 00:00:00",
    max_targets=20,
    allow_partial=True,
)
```

`report` contains:
- a local request identifier for tracing one aggregated SDK fetch
- total elapsed time and elapsed time per target
- target count
- success count
- failure count
- successful targets
- failed targets with error summaries
- requested chunk count
- successful chunk count
- failed chunk count and failed time ranges

With `allow_partial=True`, successful chunks remain available when another chunk for the same target fails. The failures remain visible in `report`.

## Plotting

Plot grouped by item:

```python
api.plot.plot_by_items(
    df,
    prefix="demo_",
    fig_label="_item",
    show_mean=True,
    overlay_earthquakes=True,
    xlabel="Time",
)
```

Plot grouped by station and point:

```python
api.plot.plot_by_stations(
    df,
    prefix="demo_",
    fig_label="_station",
    show_mean=False,
    overlay_earthquakes=False,
    ylabel="Displacement",
)
```

Plotting dependencies are loaded lazily, so importing the package without `matplotlib` is still supported when plotting is not used.

Subplot ordering rules are stable:
- `plot_by_items` sorts station subplots by `STATIONID`
- `plot_by_stations` sorts item subplots by `ITEMID`

## CLI

Initialize user config:

```bash
addereq-dm init-config --base-url http://your-api-server:8080 --app-id your_app_id --secret your_secret
```

Resolve station, point, or item:

```bash
addereq-dm resolve --station taian_center --point 1 --item vertical_z
```

Fetch data and export to file:

```bash
addereq-dm fetch \
  --station taian_center \
  --point 1 \
  --item 3123 \
  --start-time "2025-01-01 00:00:00" \
  --end-time "2025-01-01 01:00:00" \
  --kind dys \
  --allow-partial \
  --workers 2 \
  --out data.csv \
  --summary
```

CSV, JSON, and Parquet exports include a `<data-file>.metadata.json` sidecar with the query, data kind, labels, units, and fetch report. The SDK uses this sidecar automatically when the file is imported or plotted locally:

```python
from addereq_dm import export_timeseries, import_timeseries

export_timeseries(df, "data.parquet", metadata={"project": "weekly-review"})
restored = import_timeseries("data.parquet")
```

`--workers` enables controlled concurrency across independent resolved targets. It defaults to `1` and is capped at `8`; each target's 30-day windows remain sequential, and merged output keeps target order deterministic.

Plot an existing file:

```bash
addereq-dm plot --input data.csv --output-dir figures
```

Use an explicit persistent metadata snapshot instead of the export sidecar:

```bash
addereq-dm plot --input data.csv --offline-metadata --output-dir figures
```

Local files exported by the SDK restore their embedded labels and units automatically. `--offline-metadata` explicitly replaces those labels with the persistent snapshot and is the only plotting mode that reads that snapshot.

Inspect a local file without API configuration:

```bash
addereq-dm quality --input data.csv
```

Fetch and plot directly:

```bash
addereq-dm plot \
  --station taian_center \
  --point 1 \
  --item 3123 \
  --start-time "2025-01-01 00:00:00" \
  --end-time "2025-01-01 01:00:00" \
  --kind dys \
  --output-dir figures
```

Use `--kind dyu` for processed data. CP is intentionally not exposed until its upstream API is stable.

See [ROADMAP.md](ROADMAP.md) for compatibility, rate limiting, and reporting work.
