{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2d6ab5f962ca",
   "metadata": {},
   "source": [
    "# Update the electricity supply mix\n",
    "\n",
    "This workflow shows how to overwrite the electricity generation mix of a\n",
    "*database* (IOT or SUT) with up-to-date shares, either derived automatically from\n",
    "[EMBER](https://ember-energy.org/) electricity-generation data or provided\n",
    "explicitly.\n",
    "\n",
    "It applies to any database that exposes a **disaggregated power sector**,\n",
    "i.e. separate generation technologies rather than a single aggregated\n",
    "electricity sector. At the moment these are:\n",
    "\n",
    "* **EXIOBASE** (IOT and SUT, monetary), used in the examples below;\n",
    "* **EMERGING-E** (IOT);\n",
    "* any database whose electricity labels are **already aggregated** to the\n",
    "  EMBER groups — either as `electricity by <fuel>` labels or as the plain\n",
    "  EMBER names (`Coal`, `Wind`, `Other Renewables`, ...).\n",
    "\n",
    "MARIO detects the disaggregated generation labels automatically, so the same\n",
    "call works across the supported databases without extra configuration.\n",
    "\n",
    "Core methods\n",
    "------------\n",
    "\n",
    "The two main entry points are:\n",
    "\n",
    "* `Database.update_supply_mix` to rewrite the mix in place in one scenario:\n",
    "  across the intersectoral (`z`) and final-demand (`Y`) blocks on IOT\n",
    "  databases, and across `u`/`Yc` (commodity mixes) or the supply block `s`\n",
    "  (activity market shares) on SUT databases;\n",
    "* `Database.get_mix` to read back the current production mix of a sector\n",
    "  bundle as a `region -> {sector: share}` mapping (IOT only).\n",
    "\n",
    "`Database.update_supply_mix_iot` and `Database.update_mix_iot` are kept as\n",
    "backward-compatible IOT-only aliases of `update_supply_mix`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da2bcc2e",
   "source": "## What the method does\n\n`update_supply_mix` rewrites a **supply mix in place**, in one scenario,\n**without changing any column total**: within each region it only moves weight\n*between* the listed labels, so every buyer keeps its total input of the bundle\nand the rest of the table is untouched. You provide the mix either as the\nspecial string `\"electricity\"` — derived automatically from EMBER, see the IOT\nand SUT sections — or as an explicit `region -> {label: share}` mapping.\n\nWhere the weights land depends on the table type:\n\n* **IOT** — the sector rows are redistributed across the intersectoral block\n  `z` and the final demand `Y`.\n* **SUT — commodity mix (Isard)** — the same mechanic on the use block `u` and\n  `Yc`; you pass commodity labels.\n* **SUT — activity mix / market shares (Chenery-Moses)** — you pass activity\n  labels plus `commodities=...`, and the activity rows are rewritten in the\n  supply block `s` on that one commodity column, rescaled onto the market share\n  the listed activities already hold (other producers of the commodity, e.g.\n  by-product suppliers, keep theirs).\n\nThe first diagram shows the IOT case and the identical SUT commodity (Isard)\ncase; the second shows the SUT activity market-share (Chenery-Moses) case.\n\n![Supply mix on IOT and on a SUT Isard table: the sector/commodity bundle rows are redistributed across the z/Y and u/Yc buyer columns, within each region.](../../_static/images/update_supply_mix_iot_sut_isard.png)\n\n![Supply mix on a SUT Chenery-Moses table: the activity market shares are rewritten in the supply block s, on one commodity column per region.](../../_static/images/update_supply_mix_sut_chenery_moses.png)",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "id": "392bf9804cd4",
   "metadata": {},
   "source": "### The `\"electricity\"` mode\n\nThe EMBER taxonomy is more aggregated than the disaggregated electricity\nsectors exposed by EXIOBASE IOT and EMERGING-E. When you request the\n`\"electricity\"` mode, MARIO:\n\n1. aggregates the database electricity bundle to the compatible EMBER fuel\n   groups (coal, gas, nuclear, hydro, wind, solar, bioenergy, other\n   renewables, other fossil);\n2. reads the EMBER generation shares for the requested `year`;\n3. redistributes each EMBER group total back to the original database\n   sectors using the **current internal composition** of each group.\n\nThis means detailed technologies such as geothermal, tide/wave/ocean and\nsolar thermal are updated only through their parent EMBER group, while\ntransmission and distribution sectors are left unchanged."
  },
  {
   "cell_type": "markdown",
   "id": "4feec746",
   "source": "## IOT databases\n\nThe examples below parse an EXIOBASE monetary **IOT** table — which ships the\ndisaggregated electricity generation sectors — and update the mix on the `z`\nand `Y` blocks.",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "id": "aa1b3a1b8ea0",
   "metadata": {},
   "source": "### EMBER-based electricity mix\n\nUpdate the mix for one scenario. When `year` is omitted, MARIO uses the latest\nyear available in the packaged EMBER snapshot. Pass `ember_path=...` to use your\nown data: both the reduced MARIO format and the raw EMBER\n`yearly_full_release_long_format.csv` download are accepted."
  },
  {
   "cell_type": "code",
   "id": "c7959320fa81",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mario\n",
    "\n",
    "db = mario.parse_exiobase(\n",
    "    table=\"IOT\",\n",
    "    unit=\"Monetary\",\n",
    "    path=\"/path/to/IOT_2022_ixi.zip\",\n",
    "    name=\"EXIOBASE 3\",\n",
    "    year=2022,\n",
    ")\n",
    "\n",
    "# Rewrite the electricity generation mix of the baseline scenario\n",
    "db.update_supply_mix(\"electricity\", scenario=\"baseline\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20593fbee0b6",
   "metadata": {},
   "source": [
    "Inspect the resulting mix for one region. `get_mix` accepts the special\n",
    "string `\"electricity\"`, which resolves the same generation-sector bundle\n",
    "used by the update. EXIOBASE regions are two-letter codes, e.g. `\"IT\"` for\n",
    "Italy."
   ]
  },
  {
   "cell_type": "code",
   "id": "30af724233e3",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mix = db.get_mix(\"electricity\", scenario=\"baseline\")\n",
    "mix[\"IT\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f89176c3ef00",
   "metadata": {},
   "source": [
    "### Choosing the reference year\n",
    "\n",
    "Pass `year=...` to target a specific EMBER year. When a covered region has no\n",
    "observation for that year, MARIO falls back to the nearest available year for\n",
    "that region and logs the substitution."
   ]
  },
  {
   "cell_type": "code",
   "id": "4d8e392ca10f",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db.update_supply_mix(\"electricity\", scenario=\"baseline\", year=2023)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc480347e263",
   "metadata": {},
   "source": [
    "### Creating the scenario on the fly\n",
    "\n",
    "To write the updated mix into a new scenario, pass `clone_from` with an\n",
    "existing source scenario. MARIO clones it first and then applies the update."
   ]
  },
  {
   "cell_type": "code",
   "id": "9823f6baf57d",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db.update_supply_mix(\n",
    "    \"electricity\",\n",
    "    scenario=\"ember_2025\",\n",
    "    clone_from=\"baseline\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf4b4aaf68a4",
   "metadata": {},
   "source": "### Aggregated regions\n\nAggregate regions are resolved to their underlying countries before the EMBER\ngeneration is summed, so no manual bookkeeping is required. This covers two\nsituations:\n\n* **Built-in Rest-of-World regions.** EXIOBASE ships aggregate regions such as\n  `WA` (Rest of the World Asia and Pacific); MARIO expands them to their\n  packaged member countries automatically.\n* **User aggregations.** When you cluster countries yourself with `aggregate`\n  (for example into a `Rest of Europe` region), MARIO reuses the region\n  aggregation map stored on `db.meta.region_aggregation_map`.\n\nIf the map is not available (for example when the regions were renamed outside\nof `aggregate`), pass the concordance explicitly through `region_aggregation`.\nIt accepts the same inputs as `aggregate`: a Region aggregation workbook, a\npandas `Series`/`DataFrame`, or a mapping."
  },
  {
   "cell_type": "code",
   "id": "e6dafb0b9695",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db.update_supply_mix(\n",
    "    \"electricity\",\n",
    "    scenario=\"baseline\",\n",
    "    region_aggregation={\n",
    "        \"Rest of Europe\": [\"FR\", \"DE\", \"ES\", \"IT\"],\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e832c29a2d0",
   "metadata": {},
   "source": [
    "> **Note:** Regions that cannot be matched to the EMBER concordance, that are\n",
    "> missing from the EMBER snapshot, or that report no positive generation are\n",
    "> left unchanged. MARIO reports each of these cases in the logs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be6c00a696e8",
   "metadata": {},
   "source": "### Custom shares\n\nInstead of `\"electricity\"`, you can pass an explicit mapping\n`region -> {sector: share}`. The combined coefficient rows of the selected\nsectors are redistributed within each region according to the provided\nweights. Shares are expected to sum to one; pass `rescale=True` to normalise\narbitrary positive totals.\n\nThe sector labels must match those of the database. You can list the available\nelectricity generation sectors from the mix keys:"
  },
  {
   "cell_type": "code",
   "id": "9168aaad6f5a",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list(db.get_mix(\"electricity\")[\"IT\"])"
   ]
  },
  {
   "cell_type": "code",
   "id": "324a23f136c0",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shares = {\n",
    "    \"IT\": {\n",
    "        \"Production of electricity by solar photovoltaic\": 0.6,\n",
    "        \"Production of electricity by wind\": 0.4,\n",
    "    },\n",
    "}\n",
    "\n",
    "db.update_supply_mix(shares, scenario=\"baseline\", rescale=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99296a5c0eae",
   "metadata": {},
   "source": "### Exporting to EMBER groups\n\nSet `aggregate_as_ember=True` (only with `\"electricity\"`) to collapse the\ndisaggregated electricity sectors into the compatible EMBER groups right\nafter updating the mix, yielding a database whose electricity taxonomy\nmatches EMBER."
  },
  {
   "cell_type": "code",
   "id": "5def3129fb24",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db.update_supply_mix(\n",
    "    \"electricity\",\n",
    "    scenario=\"baseline\",\n",
    "    aggregate_as_ember=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10efc6b4",
   "metadata": {},
   "source": [
    "## SUT databases\n",
    "\n",
    "On SUT databases `update_supply_mix` distinguishes two cases from the labels\n",
    "you pass (disambiguate with `level='Activity'` or `level='Commodity'` when one\n",
    "label exists in both classifications):\n",
    "\n",
    "* **Commodity mix** — the commodity bundle rows are redistributed across the\n",
    "  use block `u` and the final-demand block `Yc`, exactly like the IOT case.\n",
    "* **Activity mix (market shares)** — the listed activities compete on one\n",
    "  commodity market: their rows in the supply block `s` are rewritten on the\n",
    "  columns selected by `commodities=...`. The shares are rescaled onto the\n",
    "  combined market share currently held by the listed activities, so other\n",
    "  producers of the same commodity (for instance by-product suppliers) keep\n",
    "  their share untouched.\n",
    "\n",
    "The `\"electricity\"` mode works on SUT databases too. Since EXIOBASE-style SUTs\n",
    "expose one commodity per generation technology (each produced almost 1:1 by\n",
    "its own activity), MARIO first aggregates the disaggregated electricity\n",
    "commodities into one shared `electricity` commodity — using the packaged\n",
    "profile, no mapping required — and then rewrites the market shares of the\n",
    "generation activities on that market with the EMBER-derived mix. Transmission\n",
    "and distribution commodities survive as separate commodities. Note that the\n",
    "commodity aggregation is **structural**: it affects the whole database, not\n",
    "only the selected scenario."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c053c88",
   "metadata": {},
   "source": [
    "> **Chenery-Moses tables: update the mix on every destination market.**\n",
    "> On tables converted with `to_chenery_moses`, the supply block entangles\n",
    "> technology and trade: one region's technology mix is replicated — scaled by\n",
    "> its trade shares — inside the `s` columns of every destination it exports\n",
    "> to. The default `column_regions=None` correctly rewrites the region's rows\n",
    "> in **all** destination markets, preserving each destination's trade share.\n",
    "> Restricting `column_regions` to the domestic market would leave the\n",
    "> exported production on the old mix (one fleet, two mixes — physically\n",
    "> inconsistent). Isard tables and pooled tables built with `pool_trade` are\n",
    "> immune by construction, because the technology mix is stored once.\n",
    "\n",
    "For trade mixes — the dual operation, redistributing origins per destination\n",
    "market — see [Update trade mixes](update_trade_mix.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a99e4700",
   "metadata": {},
   "outputs": [],
   "source": [
    "db_sut = mario.parse_exiobase(\n",
    "    table=\"SUT\",\n",
    "    unit=\"Monetary\",\n",
    "    path=\"/path/to/MRSUT_2022.zip\",\n",
    "    name=\"EXIOBASE 3 SUT\",\n",
    "    year=2022,\n",
    ")\n",
    "\n",
    "db_sut.update_supply_mix(\"electricity\", scenario=\"baseline\", year=2023)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28a45d7c",
   "metadata": {},
   "source": [
    "### Custom commodity aggregations and explicit mixes\n",
    "\n",
    "Pass `aggregate_commodity={'new commodity': [existing commodities]}` to build a\n",
    "custom shared market before the update, and provide the mix explicitly as\n",
    "`region -> {activity: share}` with `commodities=...` naming the target market.\n",
    "The same explicit form with commodity labels performs a commodity mix on\n",
    "`u`/`Yc` instead (no `commodities=` argument in that case)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "891c0e71",
   "metadata": {},
   "outputs": [],
   "source": [
    "# market shares: solar and wind split the market currently held by the two\n",
    "# technologies on the aggregated electricity commodity; other producers of\n",
    "# electricity (e.g. by-product suppliers) keep their share.\n",
    "db_sut.update_supply_mix(\n",
    "    {\"IT\": {\"Production of electricity by solar photovoltaic\": 0.6,\n",
    "            \"Production of electricity by wind\": 0.4}},\n",
    "    scenario=\"baseline\",\n",
    "    aggregate_commodity={\"electricity\": [\n",
    "        \"Electricity by solar photovoltaic\",\n",
    "        \"Electricity by wind\",\n",
    "        \"Electricity by coal\",\n",
    "        \"Electricity by gas\",\n",
    "    ]},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cf331bf",
   "metadata": {},
   "source": [
    "## Excel route\n",
    "\n",
    "Supply-mix updates can also be authored in the shock workbook, without writing\n",
    "any code: the `z`/`Y` sheets (IOT) and the `u`/`s`/`Yc` sheets (SUT) accept the\n",
    "`Supply mix N` type, and `shock_calc(...)` applies the mix on top of the new\n",
    "scenario. See [Shock analyses](apply_shocks.ipynb) for the sheet layouts and\n",
    "worked examples."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "mario",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}