Metadata-Version: 2.4
Name: jing
Version: 0.3.13
Summary: Download stock data and perform data analisys
Author-email: sai <dawangfei@gmail.com>
License: MIT
Keywords: jing,sai,stock
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
Requires-Dist: requests
Requires-Dist: pandas
Requires-Dist: baostock
Requires-Dist: yfinance
Requires-Dist: akshare
Requires-Dist: pyarrow
Requires-Dist: importlib-metadata; python_version < "3.10"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"

# jing

jing is a Python library for downloading stock price data to local CSV files and organizing market data.

## Installation

```bash
pip install jing
```

All required dependencies (`requests`, `pandas`, `baostock`, `yfinance`, `akshare`, `pyarrow`) will be installed automatically.

## Quickstart

```python
import jing

# Download AAPL data to local CSV
jing.D('us').download('AAPL')
```

## Main APIs

The core workflow uses the `D` class for downloading market data:

| Class | Purpose | Network? | Typical Use |
|-------|---------|----------|-------------|
| `D` | **Download** market data | Yes (API calls) | Fetch one stock or a batch list to local CSV |

### `D` — Downloader

Download stock data from Yahoo Finance, BaoStock, or AkShare into local CSV files.

```python
import jing

# US stock via Yahoo Finance
d = jing.D('us')
d.download('AAPL')

# CN stock via BaoStock (default source for CN)
d = jing.D('cn')
d.download('sz.000807')

# HK stock via AkShare
d = jing.D('hk')
d.download('00700')

# Batch download all stocks in the default CN BaoStock list
d = jing.D('cn')
d.download()  # downloads all stocks in ~/data/jing/list/cn_baostock.txt

# Batch download CN via AkShare instead
d = jing.D('cn', _source='akshare')
d.download()  # downloads all stocks in ~/data/jing/list/cn_ak.txt
```

## Data Storage

The project uses a source-first layout under `~/data/jing/` (override with `JING_DATA` env var):

```text
~/data/jing/
  raw/
    yahoo/us/
    baostock/cn/
    akshare/cn/
    akshare/hk/
  list/
    us.txt
    cn_ak.txt
    cn_baostock.txt
    hk.txt
```

Raw CSV examples:

- US Yahoo CSV: `~/data/jing/raw/yahoo/us/AAPL.csv`
- CN BaoStock CSV: `~/data/jing/raw/baostock/cn/sh.601088.csv`
- HK AkShare CSV: `~/data/jing/raw/akshare/hk/00700.csv`

### List files

List files are used only for batch downloads, for example `jing.D('us').download()` or `jing.D('cn').download()`. Single-stock downloads use the code you pass directly.

On first use, jing creates `~/data/jing/list/` and copies the bundled default lists there. Edit the files under `~/data/jing/list/`; do not edit the package files under `jing/lists/` unless you are changing the project defaults.

| Market/source | Downloader call | List file | Code format |
|---------------|-----------------|-----------|-------------|
| US Yahoo | `jing.D('us')` | `us.txt` | Yahoo ticker, e.g. `AAPL` |
| CN BaoStock (default CN) | `jing.D('cn')` | `cn_baostock.txt` | BaoStock code with exchange prefix, e.g. `sh.601088` or `sz.000807` |
| CN AkShare | `jing.D('cn', _source='akshare')` | `cn_ak.txt` | 6-digit A-share code, e.g. `601088` |
| HK AkShare | `jing.D('hk')` | `hk.txt` | 5-digit HK code, e.g. `00700` |

For CN, use `cn_baostock.txt` unless you explicitly pass `_source='akshare'`. The default `jing.D('cn')` downloader is BaoStock, and BaoStock requires exchange-prefixed codes.

Minimal examples:

`~/data/jing/list/cn_baostock.txt`

```text
sh.601088
sz.000807
sh.600519
```

`~/data/jing/list/cn_ak.txt`

```text
601088
000807
600519
```

`~/data/jing/list/us.txt`

```text
AAPL
MSFT
NVDA
```

`~/data/jing/list/hk.txt`

```text
00700
09988
03690
```

Batch download calls:

```python
import jing

jing.D('cn').download()                       # reads cn_baostock.txt
jing.D('cn', _source='akshare').download()    # reads cn_ak.txt
jing.D('us').download()                       # reads us.txt
jing.D('hk').download()                       # reads hk.txt
```

### Download cache

Downloaded data is cached as Parquet files to avoid repeated API calls on the same day:

```text
~/.cache/jing/data/
  cn_baostock_sz_000807_2025-05-25.parquet
  us_yahoo_AAPL_2025-05-25.parquet
```

Cache pattern: `<market>_<source>_<code>_<date>.parquet`

- If cache exists for today → read from Parquet (fast, ~0.01s)
- If no cache → download from API, save CSV, and write Parquet cache
- New day → automatic fresh download (old cache ignored)
- Override the cache root with `JING_CACHE`

### Data flow

#### Single stock

```mermaid
flowchart LR
    A["d.download('AAPL')"] --> B{Cache exists?}
    B -->|Yes| C[Read Parquet]
    B -->|No| D[Download from API]
    D --> E[Save CSV]
    E --> F[Write Parquet cache]
    C --> G[Return DataFrame]
    F --> G
```

#### Batch download

```mermaid
flowchart LR
    A["d.download()"] --> B[Read stock list]
    B --> C[Queue stocks]
    C --> D[Download each]
    D --> E[Print summary]
```

### Incremental Download

jing supports smart incremental downloads to save time and reduce API calls. When you already have historical data locally, jing will:

1. Fetch only new data plus an overlap period (default 5 days)
2. Check if the overlapping data matches your local CSV
3. If consistent → merge the new data
4. If inconsistent (e.g., due to dividend/split adjustments) → automatically perform a full refresh

```python
import jing

d = jing.D('us')

# Default: smart incremental download
d.download('AAPL')

# Force full refresh
d.download('AAPL', incremental=False)

# Custom overlap days and tolerance
d.download('AAPL', overlap_days=10, tolerance=0.002)
```

**Parameters:**

| Parameter | Default | Description |
|-----------|---------|-------------|
| `incremental` | `True` | Enable incremental download |
| `overlap_days` | `5` | Days to overlap for consistency check |
| `tolerance` | `0.001` | Price difference tolerance (0.1%) |

**Why overlap days?**

Stock prices are adjusted for dividends and splits. When these events occur, historical prices change. By overlapping a few days and comparing, jing can detect if adjustments have occurred and automatically refresh the entire history to maintain data consistency.

#### Incremental download flow

```mermaid
flowchart TD
    A[download\('AAPL'\)] --> B{Local CSV exists?}
    B -->|No| C[Full download]
    B -->|Yes| D[Fetch from last date - 5 days]
    D --> E{Overlapping data consistent?}
    E -->|No| C
    E -->|Yes| F[Merge new data]
    C --> G[Save CSV]
    F --> G
```

## Resuming an interrupted batch download

Batch downloads of a full market (the CN list has ~5,600 codes) are often interrupted by rate limiting. Re-running used to re-query every code, which just hit the rate limit again and never finished the tail.

Resume is **on by default**: a batch run only queries the codes whose local data is not yet current.

```python
import jing

d = jing.D('cn')

d.download()               # first run: downloads everything; gets rate limited partway
d.download()               # re-run: only downloads what is still missing
d.download(resume=False)   # force: re-query every code in the list
```

### How "already done" is decided

There is no state file to get stale — the local CSVs are the ledger:

1. Once per run, jing downloads one benchmark symbol to learn the market's latest trading date (1 API call, and usually free because the benchmark is normally already cached that day):

   | Market/source | Benchmark |
   |---------------|-----------|
   | `us` Yahoo | `AAPL` |
   | `cn` BaoStock | `sh.000001` |
   | `cn` AkShare | `600519` |
   | `hk` AkShare | `00700` |

2. Every code in the list has its local CSV's last date checked. If it already reaches that date, the code is skipped; otherwise it is downloaded.

The batch summary reports how many were skipped:

```text
最新交易日: 2026-08-31，本地数据已到达该日期的股票将被跳过
断点续传：跳过 4821 个（已是最新），剩余 854 个待下载

==================================================
           下载结果汇总
==================================================
  总计任务: 5675
  跳过（已是最新）: 4821
  本次下载: 854
  ...
```

To use a different reference symbol, edit `BENCHMARK_CODE` in `jing/resume.py`. Any actively traded symbol of that market works, since they all share one trading calendar.

### Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `resume` | `True` | Skip codes whose local data is already up to date. Batch downloads only. |
| `incremental` | `True` | Per-code incremental download (see above). |

Notes:

- `resume` is ignored when you download a single code: `d.download('AAPL')` always runs, since you asked for that stock explicitly.
- `resume` is disabled when `incremental=False`, because forcing a full refresh contradicts skipping.
- If the benchmark probe fails, resume is switched off for that run and everything is downloaded as before — a failed probe never causes missing data.
- A suspended, delisted or lagging stock never looks "up to date", so it costs one query per run. It heals itself as soon as the stock trades again.

## Configuration

### Data directory

Priority (highest first):

1. **Function call** — programmatic override
   ```python
   from jing.data_paths import set_data_root
   set_data_root('/custom/data/path')
   ```

2. **Environment variable**
   ```bash
   export JING_DATA=/custom/data/path
   ```

3. **Default** — `~/data/jing`

## Release to PyPI

1. Bump the version in `pyproject.toml`:
   ```toml
   version = "0.3.9"
   ```

2. Build the distribution packages:
   ```bash
   python -m build
   ```

3. Upload to PyPI:
   ```bash
   python -m twine upload dist/*
   ```

   > You will need a PyPI account and [API token](https://pypi.org/manage/account/token/). Configure twine once with `python -m twine upload --repository testpypi dist/*` to test first if desired.
