Metadata-Version: 2.4
Name: buildingcalibration
Version: 0.5.1
Summary: District-scale static calibration, validation and representative-district clustering for the building-energy model family
Author-email: Yassine Abdelouadoud <yassine.abdelouadoud@gmail.com>
Maintainer-email: Yassine Abdelouadoud <yassine.abdelouadoud@gmail.com>
License: The MIT License (MIT)
        =====================
        
        - Copyright © `2026` `Yoann Chiche`
        - Copyright © `2026` `Seddik Yassine Abdelouadoud`
        - Copyright © `2026` `Anna Cocchi`
        
        Permission is hereby granted, free of charge, to any person
        obtaining a copy of this software and associated documentation
        files (the “Software”), to deal in the Software without
        restriction, including without limitation the rights to use,
        copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the
        Software is furnished to do so, subject to the following
        conditions:
        
        The above copyright notice and this permission notice shall be
        included in all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
        OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
        NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
        HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
        WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
        FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
        OTHER DEALINGS IN THE SOFTWARE.
        
Project-URL: Homepage, https://git.persee.minesparis.psl.eu/planeterr/buildingcalibration
Project-URL: Repository, https://git.persee.minesparis.psl.eu/planeterr/buildingcalibration
Project-URL: Changelog, https://git.persee.minesparis.psl.eu/planeterr/buildingcalibration/-/blob/main/CHANGELOG.md
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Physics
Requires-Python: <4.0.0,>=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: buildingmodel>=1.11.0
Requires-Dist: buildingdata>=0.2.0
Requires-Dist: colorlog
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: polars!=1.43.0,!=1.44.0
Requires-Dist: geopandas
Requires-Dist: shapely
Requires-Dist: scipy
Requires-Dist: scikit-learn
Requires-Dist: fastcluster
Requires-Dist: matplotlib
Requires-Dist: seaborn
Requires-Dist: tqdm
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff>=0.16; extra == "dev"
Dynamic: license-file

# buildingcalibration

District-scale **static calibration, validation and representative-district
clustering** for the building-energy model family.

It covers the annual (non-hourly) half of the modelling chain:

- **Calibration** — the annual static calibration loop that fits a building
  stock's energy model over [`buildingmodel`](https://gitlab.com/energytransition/buildingmodel)'s
  static inference.
- **Validation** — the Eq. 5 error decomposition of simulated versus *measured*
  district consumption (ORE / Enedis), read through
  [`buildingdata`](https://gitlab.com/energytransition/buildingdata).
- **Clustering** — reduction of a national building stock to representative
  districts, plus screening of districts whose measured data cannot support
  calibration.
- **Plots** — the figures for both stages.

## Status

> **`0.2.0` is released** (tagged and on PyPI). `0.1.0` shipped the modules
> extracted from `building_eload`'s core — `core/static_simulation`,
> `core/validation`, `core/clustering.py`, `core/unreliable_districts.py`,
> `core/building_loader.py`, `utils/district_list.py`,
> `plots/static_calibration.py`, `plots/validation.py`. `0.2.0` added the two
> API items the paper-reproduction workflow was missing —
> `build_representative_districts`/`write_representative_districts` and
> `run_validation_sweep`/`average_over_years` — plus `plots.clustering`.
> See the [CHANGELOG](CHANGELOG.md) for the full history.

## Tutorials

`doc/tutorials/` has two worked examples, rebuilt against the current API
(not carried over from `building_eload`'s retired copies):

- `tutorial_01_static_simulation.ipynb` — `StaticParameters` /
  `StaticSimulation` / `StaticResults` for one district.
- `tutorial_03_validation.ipynb` — district clustering
  (`cluster_districts`, `build_representative_districts`/
  `write_representative_districts`) then `Validation` against measured
  consumption, plus `run_validation_sweep`/`average_over_years` for
  parametric studies.

The clustering steps run on synthetic data with no network needed; steps
that need real BDTOPO/census/ORE/Enedis data or a real dynamic-simulation
run are marked and left unexecuted. `building_eload`'s own
`doc/tutorials/tutorial_00_data_download.ipynb` and
`tutorial_02_dynamic_simulation.ipynb` cover reference-data acquisition and
the hourly stage these two hand off to/from.

## Quick start

### Calibrate one district (static, annual)

```python
from buildingcalibration import StaticParameters, StaticSimulation

parameters = StaticParameters(
    n_inference=5,            # stochastic building-stock draws to average over
    calibration_year=2023,
    climate_year=2023,
    output_root="results",    # -> results/static_simulation/2023/
)
results = StaticSimulation("751010101", parameters).run()
results.save_results()
```

Nothing else is needed: the building footprints (BDTOPO), the IRIS district
layer and the ORE per-IRIS annual consumption the calibration fits against are
all fetched through `buildingdata`. Pass `calibration_file=` (a frame or a path)
to pin a vintage, or `building_footprint_folder=` to use a local BDTOPO mirror.

### Fit three parameters jointly instead of one (issue #1)

The paper's Eq. 4 fits one axis, the heating setpoint. Two more knobs now exist
on `buildingmodel`'s `Parameters` — `energy_use_factor` and
`second_home_occupation_factor` — and sweeping all three exhaustively costs
`n³` district runs. `joint_calibration=True` instead sweeps each axis **one at
a time** (`3n` runs), builds a response surface from those arms and solves for
the combination minimising distance-to-measured *plus a quadratic penalty on
deviation from the defaults* — Tikhonov regularisation, i.e. the MAP estimate
under a Gaussian prior centred on the defaults, which is Eq. 4's `C_factor`
term generalised from one parameter to three:

```python
from buildingcalibration import StaticParameters, StaticSimulation, default_spec

parameters = StaticParameters(
    n_inference=5,
    calibration_year=2023,
    joint_calibration=True,          # opt-in; the default is still Eq. 4
    joint_calibration_spec=default_spec(
        residual_scale=0.05,         # assumed relative error on the measurement
        setpoint_spread=1.0,         # prior sigma, in degC
        energy_use_factor_spread=0.1,
        second_home_factor_spread=0.1,
    ),
)
simulation = StaticSimulation("751010101", parameters)
simulation.run()

print(simulation.joint_fit.describe())
```

Illustrative output (the numbers below are from the synthetic district in
`tests/calibration/test_joint_calibration.py`, over-predicting by 15 %, not from
a real run):

```
Joint calibration fit (3 fitted of 3 axes, data rank 1):
  actual_heating_set_point             18.3396  (-0.66 sigma, sensitivity +0.0527/sigma, data share 36%)
  energy_use_factor                     0.9631  (-0.37 sigma, sensitivity +0.0294/sigma, data share 11%)
  second_home_occupation_factor         0.2514  (-0.49 sigma, sensitivity +0.0388/sigma, data share 20%)
  relative gap 0.1500 (baseline) -> 0.0356 (fitted)
  verification re-run vs surrogate: <recorded by run(), see below>
  ! The data is one scalar (the district's measured annual total) against 3
    parameters, so only 1 direction is constrained by it; the rest of the fitted
    combination is the prior. ...
```

Three things about that output are load-bearing:

* **The prior spread is per parameter.** A single coefficient would trade the
  axes against each other by their raw sensitivities, so the O(0.1) factors —
  which move the district total far more per unit than a degree of setpoint
  does — absorb the whole residual while the setpoint stays pinned. `sigma`
  makes "one unit of surprise" mean the same thing on each axis. It is a stated
  assumption; `scan_regularisation_weight()` traces the fit/prior trade-off.
* **Regularisation buys well-posedness, not identifiability.** One measured
  annual total against three parameters is rank 1: the solve always returns a
  stable answer, and everything orthogonal to that one direction is the prior
  speaking. `data share` (the posterior variance reduction) is the honest number
  to report next to each fitted value — 0 % means the "fit" returned the prior
  centre. Report `joint_fit.warnings` with the values or they will be read as
  measurements.
* **The fit is verified by re-simulation.** The surface composes the
  one-at-a-time arms under an explicit assumption about how the heating effects
  combine; `run()` re-runs the demand chain at the fitted combination and
  records the discrepancy, which also leaves the shipped tables at the
  parameters that were actually fitted. `refinement_steps` feeds that residual
  back as the next Gauss-Newton step, correcting the surface's heating
  response along the baseline-to-fit direction (off by default: it converges
  on the synthetic truth of the test suite and has not been re-measured on
  real districts, and each pass costs one re-run).

### Choose how the stochastic draws enter the calibration

With `n_inference > 1` the static stage sees a *grid point × draw* table, and
reducing it to one answer is a modelling choice the two paths used to make
differently without saying so: Eq. 4 picked the single best `(point, draw)`
row, the joint fit averaged the draws first. `draw_aggregation` makes the
choice explicit and gives both paths all three options:

| `draw_aggregation` | what it does | Eq. 4 | joint |
|---|---|---|---|
| `"select_row"` | the `(point, draw)` row minimising the objective wins; for the joint fit, the best of one solve per draw. **Selects on noise** — the reported gap is biased low. | published default | |
| `"average_draws"` | average the totals over draws at each point, then rank the grid / fit the surface. The unbiased estimator of the parameter. | | default |
| `"average_fits"` | calibrate each draw on its own, then average the answers. Also reports their spread across draws (`fitted_<axis>_draw_spread`, `JointFit.draw_spread`) — the stock-stochasticity uncertainty on the fit. | | |

```python
parameters = StaticParameters(
    n_inference=10,
    calibration_year=2023,
    draw_aggregation="average_fits",   # on either path
)
```

Left unset, each path keeps its historical rule, so switching
`joint_calibration` on or off also switches the draw rule; pass
`draw_aggregation` explicitly to compare the two paths on the same estimator.
Whatever the strategy, exactly one row of `district_calibration` is flagged
`best_simulation`: the *carrier*, the already-simulated stock the shipped
tables are filtered to, at the swept point nearest the answer and — under the
averaging strategies — on the draw closest to the ensemble mean there. When
the answer is off the grid (a joint fit, or an averaged Eq. 4 setpoint) the
carrier is re-simulated at it, and the tables are tagged with the calibrated
values (`StaticSimulation.calibrated_values`). The selection is deterministic
at a fixed seed; the joint path's previous rule was not (every draw at the
nearest point tied, and `head(1)` on an unordered group picked one).

### Validate reconstructed load curves against measurements

```python
from buildingcalibration import Validation

validation = Validation(
    year=2023,
    n_clusters=20,
    scope="national",
    building_type="residential",
    input_path="results/dynamic_simulation/2023",          # hourly parquets
    output_path="results/validation/2023",
    clustering_file=clustering_frame,                      # frame or path
    unreliable_districts=unreliable_iris_frame,            # frame, ids or path
)
validation.run()
error = validation.get_error()                             # paper Eq. 5 terms
```

The measured Enedis load curves come from `buildingdata`
(`get_enedis_national()` / `get_enedis_regional()`) when
`validation_residential_file=` is left unset.

### Reduce a stock to representative districts

```python
from buildingcalibration import cluster_districts
from buildingcalibration.clustering import screen_unreliable_iris_from_parquet

unreliable = screen_unreliable_iris_from_parquet("enedis_iris_consumption.parquet")

result = cluster_districts(
    district_data,                                  # one row per district
    n_clusters=20,
    feature_columns=["heating_needs", "dhw_needs", "specific_needs"],
    unreliable_ids=unreliable["code_iris"],         # never pick these as medoids
    seed=42,
)
result.medoid_ids        # the districts to simulate
result.scaling_weights   # count-based multiplier per medoid
```

### Publish the representative-districts table the rest of the chain reads

`cluster_districts()` answers one cut at a time; the validation stage and the
medoid selectors read a single **wide** table holding several cuts side by side
(`cluster_label_<key>` / `cluster_medoid_<key>` per cut, plus the district
attributes). `build_representative_districts()` is the converter and
`write_representative_districts()` the thin writer:

```python
from buildingcalibration import write_representative_districts
from buildingcalibration.clustering import (
    cluster_districts,
    cluster_key_from_fraction,
    representative_districts_file,
)

results = {n: cluster_districts(features, n_clusters=n, seed=42) for n in (10, 20)}

write_representative_districts(
    results,
    features,                                        # one row per district
    representative_districts_file("results/clustering", scope="national"),
)
```

**Key convention** — the column suffix is always a positive integer: the
**cluster count** for a national table (`cluster_medoid_20`), the **integer
percent** of districts kept for a regional one (`cluster_medoid_5` is 5 %).
Fractions are rejected, not guessed: convert once with
`cluster_key_from_fraction(0.05) -> 5`. `representative_districts_file()` is the
single definition of the conventional file name, shared with the readers
(`get_representative_districts()`, `Validation(clustering_path=...)`), so a
table can never be written under one name and looked up under another.

### Sweep a parameter and decompose the error per variant

```python
from buildingcalibration import average_over_years, run_validation_sweep

frame = run_validation_sweep(
    {
        f"heating_threshold={value:g}": [
            {"year": year, "input_path": f"results/dynamic/{value:g}/{year}"}
            for year in (2021, 2022, 2023)
        ]
        for value in (14.0, 15.0, 16.0)
    },
    common={"n_clusters": 20, "scope": "national", "clustering_file": table,
            "unreliable_districts": unreliable},
)
# frame: long, one row per (label, year, metric) -- the full Eq. 5 terms
average_over_years(frame)   # one row per (label, metric)
```

Each case carries its own inputs (frames or explicit paths); the sweep invents
no path and writes nothing.

### Doctrine: frames first, no path registry

Nothing in this package resolves a filesystem path at import time, and there is
no `data/` tree to install:

- **Pipeline intermediates** — dynamic-simulation results, clustering tables,
  the unreliable-district list, output directories — are passed in as in-memory
  frames or as explicit paths, and have **no default**. A missing one raises a
  `ValueError` naming the parameter rather than reading from somewhere you did
  not choose.
- **External open data** — BDTOPO footprints, IRIS districts, ORE annual
  consumption, Enedis measured load curves — defaults to a
  [`buildingdata`](https://gitlab.com/energytransition/buildingdata) getter
  (`get_bdtopo()`, `get_districts()`, `get_ore()`, `get_enedis_national()` /
  `get_enedis_regional()`), which owns the download, the cache and the vintage.
  Pass a frame or a path to pin a specific vintage instead.

The one exception is the SDES *parc résidentiel* workbook read by
`plots.validation.read_sdes_data()`: `buildingdata` has no getter for it yet, so
it is a required frame-or-path argument (and reading the `.xlsx` form needs
`openpyxl`, which is not a declared dependency).

## Relationship to `building_eload`

`buildingcalibration` and `building_eload` have **no import relationship in
either direction**. `building_eload` becomes a pure hourly dynamic-simulation
library; this package owns the static/annual side. The two are coupled only by
**parquet data contracts on disk** — a calibrated stock written here is read
there, and vice versa. That seam already existed inside the old monolith; the
split just makes it a package boundary. A structural test
(`tests/test_data_path_doctrine.py`) enforces both halves of that: no path
registry, and no `building_eload` import anywhere in the package.

## The family

| Package | Role | Host |
|---|---|---|
| [`buildingdata`](https://gitlab.com/energytransition/buildingdata) | dataset access layer: BDTOPO/WFS, ERA5, INSEE census, Enedis/ORE, ELMAS | gitlab.com/energytransition |
| [`buildingmodel`](https://gitlab.com/energytransition/buildingmodel) | static inference engine: building-stock physical characteristics and annual demand | gitlab.com/energytransition |
| [`heatpumpmodel`](https://git.persee.minesparis.psl.eu/planeterr/heatpumpmodel) | shared heat-pump seasonal-performance physics (Rogeau et al. 2024) | git.persee |
| **`buildingcalibration`** | **static calibration, validation, representative-district clustering** | **git.persee** |
| [`building_eload`](https://git.persee.minesparis.psl.eu/planeterr/building_eload) | hourly dynamic simulation of district electric load | git.persee |
| [`building_eload_paper`](https://git.persee.minesparis.psl.eu/planeterr/building_eload_paper) | Snakemake reproduction workflow for the published paper; pinned to `building_eload==0.4.3` and unaffected by this split | git.persee |

## Install

```bash
pip install buildingcalibration
# or, from a checkout:
pip install -e ".[dev]"
```

Python 3.10–3.13.

## Tests

```bash
pytest                       # full suite
pytest -m "not integration"  # hermetic subset, what CI runs
```

The hermetic subset needs no data and no network. The `integration` tests run
the calibration and validation stages on real districts; they read a local
reference-data tree, `<repo>/data` by default, overridable with the
`BUILDING_ELOAD_DATA` environment variable (the name is shared with
`building_eload` on purpose, so one setting covers both checkouts).

One `integration` test — `tests/integration/test_static_to_dynamic_seam.py`,
the cross-package end-to-end seam — additionally needs **`building_eload`
installed**. It is an *optional test-time* requirement only: `building_eload` is
not a runtime dependency, not part of the `dev` extra, and the library itself
must never import it (a structural test enforces that). Install it yourself if
you want to run that test; otherwise it skips.

## Licence

MIT — see `LICENSE`.
