Metadata-Version: 2.5
Name: dlspit
Version: 2.4.0
Summary: A multi-domain experiment package: Deep Learning (MLP, CNN, ResNet, RNN, LSTM, Transformer, GAN, VAE), ML ESE Lab (regression, SVM, decision tree, random forest, XGBoost, k-means) Generative AI from scratch in NumPy (VAE, GAN, mini GPT, text-to-image, face swap, music, CycleGAN, chatbot, diffusion) and Explainable AI from scratch (SHAP, PDP, ICE, LIME, Grad-CAM, XRAI-style, Gradient x Input, LRP).
Project-URL: Repository, https://github.com/PrathamShid07/DL_Expts
Project-URL: Issues, https://github.com/PrathamShid07/DL_Expts/issues
Author: DLSPIT Team Open Source Contributor
License: MIT
Keywords: chatbot,clustering,cnn,cyclegan,decision-tree,deep-learning,diffusion,education,explainable-ai,from-scratch,gan,generative-ai,gpt,grad-cam,interpretability,kmeans,lime,lrp,lstm,machine-learning,mlp,neural-network,numpy,pytorch,random-forest,resnet,rnn,shap,sklearn,standalone-scripts,svm,tensorflow,transformer,vae,xai,xgboost
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Education
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: numpy>=1.24
Provides-Extra: all
Requires-Dist: matplotlib>=3.6; extra == 'all'
Requires-Dist: pandas>=1.5; extra == 'all'
Requires-Dist: scikit-learn>=1.3; extra == 'all'
Requires-Dist: scipy>=1.9; extra == 'all'
Requires-Dist: seaborn>=0.12; extra == 'all'
Requires-Dist: tensorflow>=2.12; extra == 'all'
Requires-Dist: torch>=2.0; extra == 'all'
Requires-Dist: xgboost>=1.7; extra == 'all'
Provides-Extra: dev
Requires-Dist: build>=1.0; extra == 'dev'
Requires-Dist: hatchling>=1.18; extra == 'dev'
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Provides-Extra: dl
Requires-Dist: tensorflow>=2.12; extra == 'dl'
Provides-Extra: genai
Requires-Dist: matplotlib>=3.6; extra == 'genai'
Requires-Dist: scikit-learn>=1.3; extra == 'genai'
Requires-Dist: torch>=2.0; extra == 'genai'
Provides-Extra: ml-ese
Requires-Dist: matplotlib>=3.6; extra == 'ml-ese'
Requires-Dist: pandas>=1.5; extra == 'ml-ese'
Requires-Dist: scikit-learn>=1.3; extra == 'ml-ese'
Requires-Dist: seaborn>=0.12; extra == 'ml-ese'
Requires-Dist: xgboost>=1.7; extra == 'ml-ese'
Provides-Extra: sklearn
Requires-Dist: scikit-learn>=1.3; extra == 'sklearn'
Provides-Extra: xai
Requires-Dist: matplotlib>=3.6; extra == 'xai'
Requires-Dist: scikit-learn>=1.3; extra == 'xai'
Requires-Dist: scipy>=1.9; extra == 'xai'
Description-Content-Type: text/markdown

# dlspit v2.4.0 — Multi-Domain Experiment Package

A pip-installable multi-domain experiment viewer and runner covering:

- **DL** — Deep Learning: MLP, Backpropagation, CNN, ResNet, RNN, LSTM,
  Transformer, GAN, VAE, Transfer Learning
- **ML-ESE** — Machine Learning ESE Lab: EDA, Linear Regression, Decision
  Tree, SVM, SVM RBF, Random Forest, XGBoost, K-Means Clustering
- **GenAI** — Generative AI from scratch in NumPy (no pretrained models, no downloads):
  VAE, GAN, mini GPT, text-to-image, face swap, music, CycleGAN, chatbot, diffusion
  → see [Generative AI experiments](#generative-ai-experiments-dlspit-genai--new-in-v210)
- **XAI** — Explainable AI from scratch in NumPy (no pretrained models, no downloads):
  regression, tree/KNN, SHAP, PDP, ICE, XRAI-style, Grad-CAM, LIME, Gradient × Input, LRP
  → see [Explainable AI experiments](#explainable-ai-experiments-dlspit-xai--new-in-v220)

For **all four domains** (`DL`, `ML-ESE`, `GENAI`, `XAI`) `dlspit` **prints the experiment** — aim, theory, formulas and the complete
code — by default, and `--run` executes it. For GenAI and XAI the printed code is a **standalone script**: copy it into Google Colab,
Jupyter or a `.py` file and it runs without this package (see [GenAI and XAI: show → copy → run](#genai-and-xai-show--copy--run)).

---

## Installation

```bash
# Core only (source viewing works for all experiments)
pip install dlspit

# With DL dependencies (tensorflow — required to --run DL experiments)
pip install "dlspit[dl]"

# With ML-ESE dependencies (scikit-learn, pandas, xgboost, …)
pip install "dlspit[ml-ese]"

# Everything
pip install "dlspit[all]"

# Showing any experiment (dlspit genai gan, dlspit xai shap, ...) needs nothing beyond the core install.
# To RUN the standalone GenAI / XAI scripts, install what each script lists under REQUIREMENTS:
pip install "dlspit[genai]"   # torch, matplotlib, scikit-learn
pip install "dlspit[xai]"     # scikit-learn, scipy, matplotlib   (Grad-CAM and XRAI also need: pip install torch)
```

---

## CLI Usage

### List commands

```bash
# Show available domains
dlspit --list

# List all DL experiments
dlspit DL --list

# List all ML-ESE experiments
dlspit ML-ESE --list
```

### Deep Learning experiments

```bash
# Print source code (default)
dlspit DL 1aMP_Xor
dlspit DL 4CNN 5Resnet 6RNN

# Execute an experiment
dlspit DL --run 1aMP_Xor
dlspit DL --run 4CNN

# As a module
python -m dl_experiments DL 1aMP_Xor
python -m dl_experiments DL --run 4CNN
```

### ML-ESE experiments

```bash
# Print source code (default)
dlspit ML-ESE linear_regression
dlspit ML-ESE decision_tree

# Execute an experiment
dlspit ML-ESE --run linear_regression
dlspit ML-ESE --run kmeans_clustering

# As a module
python -m dl_experiments ML-ESE linear_regression
python -m dl_experiments ML-ESE --run random_forest
```

---

## GenAI and XAI: show → copy → run

Every GenAI and XAI experiment is a **standalone script** that starts with its title, aim, theory and formulas and then
contains the complete code. `dlspit` prints it, exactly like it prints the ML-ESE experiments:

```bash
dlspit genai gan            # SHOW: title, aim, theory, formulas + the complete standalone code
dlspit genai gan --run      # EXECUTE the very same script
dlspit xai shap             # same for the XAI experiments
dlspit xai shap --run
dlspit genai --list         # the names (also: dlspit xai --list)
```

**Google Colab** (copy the code from the output into a cell and run it):

```python
!pip install dlspit
!dlspit genai gan                    # shows the theory and the code
# ... copy the code into a new cell and run it - nothing else from dlspit is needed.
# Colab already has torch, scikit-learn, scipy and matplotlib.  Or run it directly:
!dlspit genai gan --run
```

In a notebook you can also display it nicely, with highlighting: `from dl_experiments import show_experiment; show_experiment("genai", "gan")`
(see the next section). In **VS Code** (or any editor) save the script with `dlspit genai gan --raw > gan.py` — `--raw` prints only the file, no banner —
and run `python gan.py`.

**Why the code can be copied:** the scripts import only NumPy, scikit-learn, scipy, matplotlib and (for the neural networks) PyTorch —
never `dl_experiments`. Figures appear with `plt.show()` when you paste the code into a notebook cell or run it as a script
(`!dlspit ... --run` in Colab runs in a separate process, so paste the code into a cell to *see the figures*; the printed results always appear).
Settings you may want to change are constants at the top of each script (`SEED`, `EPOCHS`, `INSTANCE`, ...).

| GenAI command | What the script shows | `pip install` | |
|---|---|---|---|
| `dlspit genai vae` | encoder → (μ, log σ²) → reparameterisation → decoder; reconstruction + KL loss; sampling and interpolation | `torch matplotlib numpy` | |
| `dlspit genai gan` | generator, discriminator, BCE losses, two optimisers, training loop, generated samples | `torch matplotlib numpy` | |
| `dlspit genai gpt` | tokenisation, embeddings, causal self-attention, next-word loss, training, text generation | `torch numpy` | |
| `dlspit genai text-to-image` | prompt → numbers → conditional VAE that draws 8×8 pictures | `torch matplotlib numpy` | |
| `dlspit genai face-swap` | one shared encoder + two decoders; swap = A's code through B's decoder | `torch matplotlib numpy` | |
| `dlspit genai music` | next-note model with a genre input, sine-wave synthesis, WAV files, spectrum analysis | `numpy scikit-learn matplotlib` | |
| `dlspit genai cyclegan` | two generators + two discriminators, adversarial + cycle loss | `torch matplotlib numpy` | |
| `dlspit genai chatbot` | bag-of-words intent classifier with an "I did not understand" fallback | `numpy scikit-learn` | |
| `dlspit genai diffusion` | forward noising, noise-prediction training, reverse sampling | `torch matplotlib numpy` | |

| XAI command | What the script shows | `pip install` |
|---|---|---|
| `dlspit xai regression` | R², adjusted R², t-statistics, 95 % CI, odds ratios | `numpy scipy scikit-learn` |
| `dlspit xai tree-knn` | decision-tree rules, impurity vs permutation importance, KNN neighbours and vote | `numpy scikit-learn matplotlib` |
| `dlspit xai shap` | exact Shapley values and permutation-sampling approximation | `numpy scikit-learn matplotlib` |
| `dlspit xai pdp` | partial dependence and PDP importance (numerical and categorical) | `numpy scikit-learn matplotlib` |
| `dlspit xai ice` | ICE curves, PDP = mean of ICE, centred ICE | `numpy scikit-learn matplotlib` |
| `dlspit xai xrai` | Integrated Gradients → fixed-grid regions → ranking (**XRAI-style**, not full XRAI) | `torch matplotlib numpy` |
| `dlspit xai grad-cam` | gradients × feature maps → class heat map | `torch matplotlib numpy` |
| `dlspit xai lime` | perturb, weight, weighted least squares | `numpy scikit-learn matplotlib` |
| `dlspit xai gradient-input` | CBOW embeddings and Gradient × Input | `numpy scikit-learn` |
| `dlspit xai lrp` | LRP-ε, relevance conservation | `numpy scikit-learn` |

If a package is missing, `--run` says which one and prints the `pip install` line. All scripts use small synthetic data (nothing is downloaded,
no pretrained model is used), fixed random seeds, and finish in roughly 1–15 seconds on a laptop CPU (no GPU needed; the first run is slower while PyTorch loads).

**Also available:** the original parameterised implementations that need only NumPy (hand-written back-propagation, `--epochs`, `--samples`, ... ,
and the Python API `dl_experiments.genai.vae.run_experiment(...)`) are still there: add `--library`, e.g.
`dlspit genai vae --library --epochs 50` or `dlspit xai pdp --library --feature 1 --plot`; `dlspit genai vae --library --source` prints that code.
The sections "Generative AI experiments" and "Explainable AI experiments" further below describe this library version (and contain
viva questions and exam modifications that apply to both).

---

## Read an experiment in Google Colab / Jupyter

Install the package in a notebook and call one function. The cell output shows the experiment's **theory** followed by its
**complete Python source code** (syntax-highlighted), exactly as stored in the package. Nothing is executed and nothing is
rewritten: the theory is the header docstring of the experiment's `.py` file (GenAI, XAI: title, aim, theory, formulas) or its leading
`# THEORY` comment block (DL, ML-ESE), and the code is the file itself. For GenAI and XAI that file is the standalone script, so the
code you see runs when you paste it into a cell.

```python
!pip install dlspit

from dl_experiments import show_experiment

show_experiment("genai", "chatbot")          # theory + full source of the chatbot experiment
show_experiment("xai", "shap")               # domains: "genai", "xai", "dl", "ml-ese"
show_experiment("dl", "4CNN")
show_experiment("ml-ese", "decision_tree")
```

Useful options:

```python
show_experiment("xai", "grad-cam", code=False)     # theory only
show_experiment("xai", "grad-cam", theory=False)   # code only
show_experiment("xai", "lrp", output="text")       # force plain text (default: HTML in a notebook, text in a terminal)
```

The valid names are listed by `dl_experiments.genai.list_experiments()`, `dl_experiments.xai.list_experiments()`,
`dl_experiments.dl.list_experiments()` and `dl_experiments.ml_ese.list_experiments()`; the same aliases as the command line work
(`"gradcam"`, `"text_to_image"`, ...). An invalid name raises an error that lists the valid ones.

To get the text itself (for example to copy it into a journal, or to print it):

```python
from dl_experiments import get_theory, get_source

print(get_theory("genai", "chatbot"))        # the explanation only
print(get_source("genai", "chatbot"))        # the complete file
```

Syntax highlighting uses Pygments, which Colab and Jupyter already include (without it the code is shown as plain monospaced
text); no extra package is required. To *run* an experiment instead of reading it, use `dlspit genai chatbot` or
`from dl_experiments.genai.chatbot import run_experiment`.

---

## DL Experiment Index

| Key                  | Description                                         |
|----------------------|-----------------------------------------------------|
| `1aMP_Xor`           | McCulloch-Pitts neuron — XOR & XNOR gates           |
| `1bMP_Binary`        | MLP binary classifier (XNOR via backprop)           |
| `2aLoss_Activation`  | Activation & loss function comparison on MNIST      |
| `2bBPN`              | Backpropagation on MNIST (SGD feedforward NN)       |
| `3regularization`    | L2 regularisation + Dropout on MNIST                |
| `4CNN`               | Convolutional Neural Network on MNIST               |
| `5Resnet`            | ResNet50 transfer learning on CIFAR-10              |
| `6RNN`               | Simple RNN for IMDB sentiment analysis              |
| `7Lstm`              | Stacked LSTM for time-series prediction             |
| `8Transform`         | Transformer encoder for IMDB text classification    |
| `9aVAE`              | Variational Autoencoder on MNIST                    |
| `9bGAN`              | Generative Adversarial Network on MNIST             |
| `10Transfer`         | MobileNetV2 transfer learning on CIFAR-10           |

---

## ML-ESE Experiment Index

| Key                  | Description                                                 | Dataset           |
|----------------------|-------------------------------------------------------------|-------------------|
| `eda_titanic`        | EDA & data visualisation on the Titanic dataset             | Seaborn built-in  |
| `linear_regression`  | Linear Regression — predicting student marks                | `dataset.csv`     |
| `decision_tree`      | Decision Tree Classifier — sports match prediction          | Inline            |
| `svm_classifier`     | SVM Classifier — student pass/fail prediction               | Inline            |
| `svm_rbf_kernel`     | SVM with RBF Kernel — non-linear XOR-like classification    | Inline            |
| `random_forest`      | Random Forest Classifier — loan approval prediction         | `loan_approval_dataset.csv` |
| `xgboost_classifier` | XGBoost Classifier — loan approval prediction               | `loan_approval_dataset.csv` |
| `kmeans_clustering`  | K-Means Clustering — customer segmentation (Elbow Method)   | `seg.csv`         |

> **Note**: Experiments marked with a CSV dataset require that file to be
> present in your **current working directory** when using `--run`.
> Source code display (`dlspit ML-ESE <key>`) always works without any CSV.

---

## Python API

```python
# DL domain
from dl_experiments.dl import list_experiments, show_source, run_experiment

print(list_experiments())          # ['1aMP_Xor', '1bMP_Binary', ...]
show_source("4CNN")                # print source, never executes
run_experiment("4CNN")             # executes (requires tensorflow)

# ML-ESE domain
from dl_experiments.ml_ese import list_experiments, show_source, run_experiment

print(list_experiments())          # ['eda_titanic', 'linear_regression', ...]
show_source("linear_regression")   # print source, never executes
run_experiment("decision_tree")    # executes (requires scikit-learn)
```

---

## Requirements

| Group    | Packages                                          | Required for               |
|----------|---------------------------------------------------|----------------------------|
| core     | `numpy>=1.24`                                     | Always                     |
| `dl`     | `tensorflow>=2.12`                                | Running DL experiments     |
| `ml-ese` | `scikit-learn`, `pandas`, `matplotlib`, `seaborn`, `xgboost` | Running ML-ESE experiments |
| *(none)* | NumPy only (core)                                 | Showing any experiment; running the `--library` GenAI / XAI versions |
| `genai`  | `torch`, `matplotlib`, `scikit-learn`             | Running the standalone GenAI scripts (`--run`, or paste them) |
| `xai`    | `scikit-learn`, `scipy`, `matplotlib`             | Running the standalone XAI scripts (Grad-CAM, XRAI also need `torch`) |

---

# Generative AI experiments (`dlspit genai`) — new in v2.1.0

> **Which version is this section about?** `dlspit genai <name>` now shows the **standalone script** (PyTorch / scikit-learn; see
> [show → copy → run](#genai-and-xai-show--copy--run)). This section describes the original **NumPy-only library version**
> (`--library`, Python API `dl_experiments.genai.<module>`), including the educational substitutions, viva questions and exam
> modifications, which apply to both. Function names in the "code walk-through" lines refer to the library modules.

Nine Generative AI experiments, implemented **from scratch in Python + NumPy** (library version) and designed for
college practical examinations:

* **No pretrained models** — nothing is loaded from TensorFlow Hub, Hugging Face, PyTorch Hub, an API, …
* **No dataset downloads** — every experiment builds a small synthetic dataset in code.
* **No internet, no API keys, no GPU** — each experiment runs in seconds on a laptop.
* **No TensorFlow / PyTorch needed (library version)** — the networks use a tiny NumPy toolkit with hand-written back-propagation
  (`dl_experiments/genai/_nn.py`, about 350 lines, gradient-checked in the tests).
* **Short, commented code** — one file per experiment, split into *data → model → training → demo*.

> **Honest scope.** These are *educational* versions. They show how each algorithm works on toy data; they do **not**
> reproduce the quality of GPT-2, DALL·E, Stable Diffusion, Jukebox, etc. Each module's docstring states the
> substitution it makes. Training from scratch is allowed; loading pretrained weights is not.

## A. Installation

```bash
pip install dlspit            # enough for the --library versions and for showing any experiment (NumPy only)
pip install "dlspit[genai]"   # to RUN the standalone scripts: torch, matplotlib, scikit-learn

# optional extras of the other domains are unchanged:
pip install "dlspit[dl]"      # TensorFlow, for the original DL experiments
pip install "dlspit[ml-ese]"  # scikit-learn, pandas, ... for the ML-ESE experiments
pip install "dlspit[all]"     # everything
```

There is intentionally **no** `dlspit[genai]` extra: the GenAI experiments have no dependency beyond NumPy.
The base installation therefore stays small, and `import dl_experiments` never imports TensorFlow, scikit-learn, etc.

## B. The experiments

The numbering follows the PDFs in `Genai/Expt Done Docs/`.

| # | Experiment | Algorithm demonstrated | Python module | CLI command |
|---|------------|------------------------|---------------|-------------|
| 1 | Variational Autoencoder | encoder → (μ, log σ²) → reparameterisation → decoder; reconstruction + KL loss | `dl_experiments.genai.vae` | `dlspit genai vae` |
| 2 | GAN | generator vs discriminator, non-saturating loss, Fréchet distance | `dl_experiments.genai.gan` | `dlspit genai gan` |
| 3 | GPT-style language model | token + position embeddings, causal self-attention, pre-train → fine-tune, top-k/top-p sampling | `dl_experiments.genai.mini_gpt` | `dlspit genai gpt` |
| 4 | Text-to-image | prompt → numbers → conditional generator | `dl_experiments.genai.text_to_image` | `dlspit genai text-to-image` |
| 5 | Face swapping | landmarks → align → mask → colour match → blend; shared encoder + two decoders | `dl_experiments.genai.face_swap` | `dlspit genai face-swap` |
| 6 | Music generation | autoregressive note model with genre conditioning, WAV synthesis, audio analysis | `dl_experiments.genai.music_generation` | `dlspit genai music` |
| 7 | CycleGAN | two generators + two discriminators, adversarial + cycle (+ identity) loss | `dl_experiments.genai.cyclegan` | `dlspit genai cyclegan` |
| 8 | Chatbot | bag-of-words intent classifier with an "unknown" fallback | `dl_experiments.genai.chatbot` | `dlspit genai chatbot` |
| 9 | Diffusion (DDPM) | forward noising, noise prediction, reverse sampling, schedules, guidance | `dl_experiments.genai.diffusion` | `dlspit genai diffusion` |

**Dependencies of the library versions: none** (NumPy only). The standalone scripts need the packages listed in the table above.

### Educational substitutions (what the lab document uses → what is implemented here)

| # | Document | This package |
|---|----------|--------------|
| 1 | MNIST digits, Keras | 8×8 synthetic images with one bright blob at a random position (2 hidden factors → a 2-D latent space you can see) |
| 2 | MNIST digits, PyTorch, FID/IS | 2-D "ring" (or Gaussian) point cloud; `frechet_distance` uses the FID formula on the points |
| 3 | Fine-tune pretrained **DistilGPT2** | a ~18 k-parameter GPT trained from scratch. "Fine-tuning" is shown by pre-training on a tiny *general* corpus and then continuing on a tiny *AI* corpus. This is **not** GPT-2 and the pre-training is ours |
| 4 | Hosted **DALL·E Mini / Craiyon** API | a conditional VAE that draws 8×8 shapes from prompts such as `"bright square left"`. It cannot draw a cat |
| 5 | LFW face photos, Keras conv autoencoder | synthetic cartoon faces with **manually supplied** landmarks (no face detector) |
| 6 | Audio produced by **OpenAI Jukebox** | tiny genre-conditioned melody model + sine-wave synthesiser; the same audio analysis as the document |
| 7 | Horse2Zebra photos, ResNet generator | two synthetic 8×8 image domains (bright square on dark ↔ dark square on bright), small convolutional networks |
| 8 | **DistilGPT2** chatbot (generative) | retrieval chatbot: a neural intent classifier that picks a pre-written answer. **Not** a generative LLM |
| 9 | PyTorch DDPM on moons; **Stable Diffusion** in Colab | NumPy DDPM on moons (Parts 1–2); classifier-free guidance on a 2-D class label as the toy stand-in for Part 3 |

## C. Usage

### Command line

```bash
dlspit genai --list                              # the nine experiments
dlspit genai vae                                 # SHOW the standalone script (theory + code)
dlspit genai vae --run                           # run that script
dlspit genai vae --library                       # RUN the NumPy-only library version
dlspit genai gan --library --epochs 100          # ... and change a parameter (options need --library)
dlspit genai diffusion --library --samples 1000 --seed 3
dlspit genai music --library --save-dir out      # also write WAV / PNG files into ./out
dlspit genai gpt --library --source              # print the LIBRARY source code
dlspit genai chatbot --library                   # interactive chat (library); type 'exit' to stop
dlspit genai chatbot --library --demo            # scripted conversation instead
python -m dl_experiments genai vae               # same thing without the console script
```

Names are case-insensitive and accept `_`/`-` and a few aliases (`mini_gpt`, `text_to_image`, `faceswap`, `ddpm`, or the
experiment number `1`–`9`). Options: `--epochs`, `--samples` (size of the synthetic training set), `--seed`,
`--save-dir`, `--demo` (chatbot only), `--source`. These options apply to the library version, so they require `--library`
(without it the command explains this and exits). An option an experiment does not have (for example `--samples` for `gpt`)
produces a clear error. The standalone scripts have their settings as constants at the top of the file.

### Python

Every module has a `run_experiment(...)` with working defaults that trains, prints a report and returns a dictionary.

```python
from dl_experiments.genai.vae import run_experiment
results = run_experiment(epochs=40)              # results["reconstructions"], results["samples"], ...
```

```python
from dl_experiments.genai.gan import run_experiment
results = run_experiment(epochs=100, kind="ring")   # results["fake"], results["frechet_after"]
```

```python
from dl_experiments.genai.mini_gpt import run_experiment, generate
results = run_experiment(epochs=15)
print(generate(results["tuned_model"], results["tokenizer"], "machine learning is",
               temperature=0.7, top_k=5))
```

```python
from dl_experiments.genai.text_to_image import run_experiment, generate_images
results = run_experiment(epochs=60)
pictures = generate_images(results["model"], "dark bar right", n=4)   # (4, 8, 8) array
```

```python
from dl_experiments.genai.face_swap import run_experiment
results = run_experiment(epochs=60)            # results["pipeline"]["result"], results["swapped"]
```

```python
from dl_experiments.genai.music_generation import run_experiment
results = run_experiment(save_dir="out")       # writes out/generated_jazz.wav, out/generated_rock.wav
```

```python
from dl_experiments.genai.cyclegan import run_experiment
results = run_experiment(epochs=60, lambda_cycle=10.0)
```

```python
from dl_experiments.genai.chatbot import Chatbot, run_chatbot
bot = Chatbot().train()
print(bot.respond("what is a gan")[0])
run_chatbot(bot)                               # interactive terminal chat
```

```python
from dl_experiments.genai.diffusion import run_experiment
results = run_experiment(epochs=100, T=50, schedule_kind="cosine", study=False, guidance=False)
```

The registry gives the same access by name: `from dl_experiments.genai import list_experiments, run_experiment, show_source`.

### Notes

* **Reproducibility.** Every experiment takes `seed`; the same seed gives the same result. GAN-type models (Experiments 2 and 7) are
  sensitive to the seed — if a run looks poor, try another `--seed` and read the printed pictures, not just the loss.
* **Saving results.** `--save-dir DIR` (or `save_dir="DIR"`) writes PNG images (no Pillow needed) and, for music, WAV files.
* **Tests.** From the repository root: `pip install pytest && pytest -q` (≈ 5 s, offline; the tests train tiny models, check shapes, losses,
  causal masking, noise schedules, the CLI, and compare every hand-written backward pass with finite differences).
* **macOS + NumPy 2.2.x.** You may see `RuntimeWarning: ... encountered in matmul`. This is a known NumPy/Accelerate quirk that fires even
  for ordinary finite matrices; results are unaffected, and upgrading NumPy removes it.

## D. Algorithm explanations and E. Practical-exam preparation

Each experiment below has its explanation (D) and its exam notes (E). Click a heading to expand it.
Shared helper code lives in `_nn.py` (layers with `forward`/`backward`, `Adam`, losses) and `_viz.py` (ASCII pictures,
PNG writer). Everything else is inside the experiment's own file.

<details>
<summary><b>Experiment 1 — Variational Autoencoder (<code>vae.py</code>)</b></summary>

**What it does.** Learns to compress an image to a few numbers (the *latent code*) and to rebuild it, in such a way that
random latent codes also decode to plausible *new* images.

**Components.** `VAE.encode` (image → mean μ and log-variance log σ²), `VAE.reparameterize` (z), `VAE.decode`
(z → pixel probabilities), `reconstruction_loss`, `kl_divergence`, `train_step`.

**Maths.**
`z = μ + σ·ε, ε ~ N(0, I)` (reparameterisation: randomness is moved into ε so gradients can flow through μ and σ);
`loss = BCE(x, x̂) + kl_weight · KL`, with `KL = −½ Σ (1 + log σ² − μ² − σ²)`, which pulls each code towards N(0, I) so that
sampling `z ~ N(0, I)` makes sense.

**Training.** Mini-batch Adam; the backward pass is written by hand in `VAE.train_step`.

**Run.** `dlspit genai vae` or `run_experiment(epochs=60)`.

**Expected output.** Loss falls from ≈ 42 to ≈ 14; original-vs-reconstructed blobs look alike (MSE ≈ 0.01); samples from
N(0, I) are blobs at different places; the interpolation row shows a blob moving smoothly. The two latent axes correlate
(|r| ≈ 0.9) with the blob's x and y position.

**Parameters.** `epochs`, `n_samples`, `image_size`, `latent_dim`, `kl_weight` (`0` removes the KL term = a plain
autoencoder, Part 1 of the document; large values give blurrier but smoother latent spaces), `seed`.

**Limits.** 8×8 blobs only; fully connected layers; no colour.

**Exam notes**
* *Summary.* Encoder gives a Gaussian; sample z with the reparameterisation trick; decoder rebuilds; loss = reconstruction + KL.
* *Code walk-through.* `make_dataset` → `VAE.__init__` (layers) → `encode` → `reparameterize` → `decode` → losses → `train_step`
  (forward, hand-written backward, `optimizer.step`) → `train_vae` (epoch loop) → `generate_samples` / `interpolate`.
* *Key functions.* `reparameterize`, `kl_divergence`, `train_step`.
* *Likely modifications.* change `latent_dim`; change `kl_weight` (β-VAE); add a second hidden layer in `VAE.__init__`;
  use MSE instead of BCE; generate 16 samples; plot the loss curve (`history["total"]`); interpolate between two other images.
* *Viva.*
  1. *Why the reparameterisation trick?* Sampling is not differentiable; writing z = μ + σε makes z a differentiable function of μ and σ.
  2. *What does the KL term do?* Regularises the latent distribution towards N(0, I) so random codes decode to valid images.
  3. *Autoencoder vs VAE?* An autoencoder maps to a point and has gaps in latent space; a VAE maps to a distribution, giving a smooth space you can sample from.
  4. *Why predict log σ² instead of σ?* It can be any real number and σ = exp(½ log σ²) is always positive.
  5. *What happens with `kl_weight=0`?* Good reconstructions but a latent space that is not organised for sampling.
</details>

<details>
<summary><b>Experiment 2 — GAN (<code>gan.py</code>)</b></summary>

**What it does.** A generator turns random noise into samples; a discriminator tries to tell them from real data. Training
them against each other pushes the generator towards the real distribution.

**Components.** `create_models` (two MLPs), `train_step` (discriminator update, then generator update), `train_gan`,
`generate_samples`, `frechet_distance`.

**Maths.** D maximises `log D(x) + log(1 − D(G(z)))`. G uses the *non-saturating* loss `−log D(G(z))`. The discriminator
outputs a logit; `sigmoid(logit)` = P(real). `frechet_distance` = `|μ₁−μ₂|² + Tr(C₁ + C₂ − 2(C₁C₂)^½)` (the FID formula applied to the points).

**Training.** Each batch: (1) train D on real=1/fake=0; (2) train G so that D says real for fakes. Adam with β₁ = 0.5.

**Run.** `dlspit genai gan` or `run_experiment(epochs=150, kind="ring")`.

**Expected output.** Real ring and generated ring drawn in ASCII; D loss hovering around 1.2–1.4 (= 2 ln 2 ≈ 1.39 is "D is guessing");
Fréchet distance dropping from ≈ 2–3 to well below 0.1 for the default ring.

**Parameters.** `epochs`, `n_samples`, `kind` (`"ring"` or `"gaussian"`), `latent_dim`, `seed`. Learning rate and batch size are arguments of `train_gan`.

**Limits.** 2-D data. GANs are unstable: other seeds/`kind="gaussian"` give less accurate matches and mode collapse is possible.

**Exam notes**
* *Summary.* Two networks play a minimax game; at equilibrium D outputs ½ everywhere.
* *Code walk-through.* `make_real_data` → `create_models` → `train_step` (D step, then G step through D) → `train_gan` → `frechet_distance`.
* *Key functions.* `train_step`, `bce_with_logits` (in `_nn.py`).
* *Likely modifications.* different dataset (`kind`), bigger `latent_dim`, change learning rate, train D twice per G step, use LeakyReLU slope 0.1, print losses every epoch, compare samples at epochs 10/50/150.
* *Viva.*
  1. *Roles of G and D?* G creates fakes from noise; D classifies real vs fake.
  2. *Why train G with "fake labelled as real"?* The non-saturating loss gives strong gradients early on, when D easily rejects fakes.
  3. *What is mode collapse?* G outputs only a few kinds of samples, ignoring other parts of the data.
  4. *Why can't we use only loss curves to judge a GAN?* Losses oscillate; use samples and metrics such as FID/IS.
  5. *What does the Fréchet distance measure?* The distance between Gaussians fitted to real and generated features.
</details>

<details>
<summary><b>Experiment 3 — Tiny GPT: language model, fine-tuning ideas, sampling (<code>mini_gpt.py</code>)</b></summary>

**What it does.** A decoder-only Transformer learns to predict the next word of a small text, then generates new text one
word at a time. **It is not GPT-2 or DistilGPT2** — no pretrained weights are used.

**Components.** `Tokenizer` (word ↔ id, `<unk>` = 0), `make_sequences` (inputs and next-token targets), `MiniGPT`
(token + position embeddings → `TransformerBlock` → final `LayerNorm` → linear head), `CausalSelfAttention`, `LayerNorm`,
`train_model`, `evaluate` (loss, perplexity), `sample_next_token` (temperature, top-k, top-p), `generate`.

**Maths.** `attention = softmax(QKᵀ/√d + causal mask)·V`, where the mask sets scores for *future* positions to −∞.
Block: `h = x + Attn(LN(x))`, `out = h + FFN(LN(h))`. Loss = cross-entropy of the next token; `perplexity = exp(loss)`.
Tokenisation (text → ids) and embedding (id → vector of numbers) are separate steps; the run prints both.

**Training / "fine-tuning".** Stage A trains from random weights on the AI corpus. Stage B first trains on a tiny general
corpus ("pre-training"), then continues on the AI corpus with a lower learning rate ("fine-tuning"). In a real project the
pre-training would be done by someone else on billions of words; here it is only a small demonstration of the *procedure*.

**Run.** `dlspit genai gpt` or `run_experiment(epochs=15)` (≈ 2 s).

**Expected output.** Vocabulary of ≈ 137 words; validation perplexity falling from ≈ 5–14 to ≈ 2; text such as
*"artificial intelligence is used in medicine and education."*; five sampling strategies side by side. In this toy setting the
"pre-train + fine-tune" curve ends **about as good as training from scratch** (not clearly better) — pre-training helps
when it is large and related to the target domain, which two 20-sentence corpora are not.

**Parameters.** `epochs`, `block_size`, `d_model`, `seed`; in `generate`: `max_new_tokens`, `temperature`, `top_k`, `top_p`.
Change the corpora through `AI_FACTS` / `GENERAL_FACTS` at the top of the file.

**Limits.** Single attention head, one layer, word-level vocabulary, ~18 k parameters; it mostly memorises its facts.

**Exam notes**
* *Summary.* Embed tokens + positions → masked self-attention → feed-forward → logits; train on next-word prediction; sample autoregressively.
* *Code walk-through.* `Tokenizer` → `make_sequences` → `MiniGPT.forward` → `loss_and_backward` → `train_model` → `sample_next_token` → `generate`.
* *Key functions.* `CausalSelfAttention.forward` (look at the `np.where(causal_mask(...), -1e9, scores)` line), `sample_next_token`.
* *Likely modifications.* different corpus; change `block_size`/`d_model`; two layers (`n_layers=2`); try temperature 0.2 vs 1.5; top-k=3; plot perplexity vs epoch (`history["val_ppl"]`); add a new fact to `AI_FACTS`.
* *Viva.*
  1. *Why a causal mask?* The model must predict token t from tokens < t only; otherwise it could copy the answer.
  2. *Tokenisation vs embedding?* Tokenisation maps text to integer ids; the embedding table maps each id to a learned vector.
  3. *Why positional embeddings?* Attention itself is order-blind; positions tell the model where each token is.
  4. *Effect of temperature / top-k / top-p?* Temperature sharpens (<1) or flattens (>1) probabilities; top-k keeps the k best tokens; top-p keeps the smallest set with cumulative probability ≥ p.
  5. *Training from scratch vs fine-tuning?* From scratch starts from random weights and needs lots of data; fine-tuning starts from pretrained weights and adapts them to a domain with little data and a small learning rate.
  6. *What is perplexity?* `exp(cross-entropy)`; roughly the number of tokens the model is "choosing between" at each step.
</details>

<details>
<summary><b>Experiment 4 — Text-to-image (<code>text_to_image.py</code>)</b></summary>

**What it does.** Turns a short text prompt into a picture by conditioning an image generator on numbers made from the prompt.
It is a miniature of the DALL·E idea, **not** DALL·E: vocabulary of 7 words, 8×8 pictures of a square/cross/bar that is
bright/dark and on the left/right.

**Components.** `tokenize_prompt` → `encode_prompt` (multi-hot "text embedding") → `create_model` (a *conditional VAE* from
`vae.py`) → `train_model` → `generate_images` → `describe_image` / `prompt_match_rate` (read the picture back and check it).

**Maths.** The prompt vector `c` is concatenated to the encoder input and to the decoder input; the loss is the VAE loss.
Generation: `image = decoder(z, c)` with `z ~ N(0, I)`. During training each prompt word is hidden with probability 0.3
(`word_dropout`), so *incomplete* prompts also work — and then the noise `z` decides the missing attributes.

**Run.** `dlspit genai text-to-image` or `run_experiment(epochs=120)`.

**Expected output.** For `"bright square left"`, `"dark bar right"` about 80–95 % of the generated pictures match the prompt; for
vague prompts (`"square"`, `"dark"`) the pictures vary in the unspecified attributes; the held-out prompt `"dark cross right"`
(never trained) works only partly (roughly 40–80 %) — an honest example of limited generalisation.

**Parameters.** `epochs`, `n_samples`, `held_out` (prompts removed from training), `seed`; `word_dropout` in `train_model`.

**Limits.** Fixed tiny vocabulary; no learned text encoder; the picture is 64 pixels.

**Exam notes**
* *Summary.* Text → vector → condition both encoder and decoder → sample z → image.
* *Code walk-through.* `encode_prompt` → `make_dataset` → `create_model` → `train_model` (word dropout) → `generate_images` → `prompt_match_rate`.
* *Key functions.* `encode_prompt`, `generate_images`.
* *Likely modifications.* add a word (e.g. a new shape in `draw_image` and `SHAPES`); change `word_dropout`; hold out another prompt; larger `latent_dim`.
* *Viva.*
  1. *What is conditioning?* Giving the model extra information (here the prompt) so the output depends on it.
  2. *Why do two generations from the same prompt differ?* The random noise z changes details the prompt does not fix.
  3. *Why do more detailed prompts give more predictable images?* They constrain more attributes, leaving less to the random noise.
  4. *How does real DALL·E differ?* Learned Transformer text encoders, huge image–text datasets, and image tokens or diffusion as the generator.
  5. *Limitations?* Cannot draw anything outside its training vocabulary; may mis-compose unseen combinations.
</details>

<details>
<summary><b>Experiment 5 — Face swapping (<code>face_swap.py</code>)</b></summary>

**What it does — two methods on synthetic cartoon faces with manually supplied landmarks (no face detector).**

*Method 1, image-processing pipeline* (`swap_face`): `estimate_similarity` (least-squares scale + rotation + translation from
the 3 landmarks) → `warp_image` (backward mapping, bilinear sampling) → `face_mask` (soft ellipse placed from the landmarks) →
`match_color` (per-channel mean/std inside the mask) → `blend` (`mask·face + (1−mask)·background`).

*Method 2, shared encoder + two decoders* (`FaceSwapAutoencoder`, the idea in the lab PDF): one encoder compresses any face;
decoder A is trained only on person A, decoder B only on person B. **Swap = encode A, decode with B.** The encoder has to keep
pose/expression (shared), the decoders keep identity (colour of skin and eyes).

**Maths.** Similarity transform `x' = a·x − b·y + tx, y' = b·x + a·y + ty` solved by least squares; loss for Method 2 is MSE per identity.

**Run.** `dlspit genai face-swap` or `run_experiment(epochs=150)`.

**Expected output.** Landmark alignment error ≈ 1e-14 px; mask and swapped picture in ASCII; for the autoencoder, rows
*original A / reconstructed A / swapped A→B / truth (B with A's pose)*. The swapped faces have B's colours with A's
head position and mouth; the swap-to-truth error is about 10× smaller than swap-to-original.

**Parameters.** `epochs`, `n_samples`, `seed`; `feather`, `color_correct` in `swap_face`.

**Limits.** Cartoon faces on 16×16/32×32 canvases; fully connected autoencoder; landmarks are given, not detected.
This is a demonstration of the mechanism, not a deepfake tool.

**Exam notes**
* *Summary.* Method 1 = geometry + blending; Method 2 = shared encoder, per-person decoders, cross-wiring at test time.
* *Code walk-through.* `draw_face` → `estimate_similarity` → `warp_image` → `face_mask` → `match_color` → `blend` → `swap_face`;
  then `make_identity_dataset` → `FaceSwapAutoencoder` → `train_autoencoder` → `swap`.
* *Key functions.* `estimate_similarity`, `blend`, `FaceSwapAutoencoder.swap`.
* *Likely modifications.* remove colour matching (`color_correct=False`) and compare; change `feather`; use a different `code_dim`; swap B→A.
* *Viva.*
  1. *Why a shared encoder?* It is forced to store only general structure (pose/expression); identity details end up in the separate decoders.
  2. *How does the swap work?* Pass A's code to decoder B.
  3. *Why align faces first?* So eyes and mouth coincide, otherwise the blended face looks distorted.
  4. *Why a soft mask?* A hard edge leaves a visible seam; feathering blends smoothly.
  5. *Ethical concerns?* Deepfakes can be used for misinformation or harassment; consent and disclosure matter.
</details>

<details>
<summary><b>Experiment 6 — Music generation (<code>music_generation.py</code>)</b></summary>

**What it does.** The lab PDF analyses audio from OpenAI Jukebox. Here a tiny autoregressive model generates **note tokens**
conditioned on a genre, the notes are turned into a waveform, and the same kind of audio analysis is computed.

**Components.** `make_melodies` (two hand-written "composers": jazz = smooth steps in a Dorian scale, rock = repeated minor-pentatonic
notes), `MelodyModel` (previous two notes + genre one-hot → next note), `synthesize` (sine harmonics with an envelope, tanh distortion for
rock), `save_wav` (standard-library `wave`), `analyze_audio` (duration, RMS, crest factor, spectral centroid via FFT).

**Maths.** Next-note cross-entropy; frequency `f = 261.63·2^(semitones/12)`; crest factor `20·log10(1/RMS)` of the peak-normalised
signal; spectral centroid `Σ f·|X(f)| / Σ |X(f)|`.

**Run.** `dlspit genai music --save-dir out` or `run_experiment(save_dir="out")`.

**Expected output.** Two melodies printed as note names, loudness envelopes, a jazz-vs-rock table: rock has more repeated notes,
higher RMS and spectral centroid (≈ 1.5 kHz vs ≈ 0.5 kHz) and lower crest factor. Two playable 16-bit mono WAV files if `save_dir` is given.

**Parameters.** `epochs`, `n_samples` (melodies per genre), `length`, `sample_rate`, `seed`; `temperature` in `MelodyModel.generate`.

**Limits.** Monophonic, 14 tokens, sine-wave sound; it learns the statistics of the two rule-based composers, not real music.

**Exam notes**
* *Summary.* Notes are tokens; an autoregressive model samples the next token given the past and the genre; tokens → sound → measurements.
* *Code walk-through.* `make_melodies` → `MelodyModel.features/train/generate` → `synthesize` → `analyze_audio` → `save_wav`.
* *Key functions.* `MelodyModel.generate`, `analyze_audio`.
* *Likely modifications.* add a third genre (add to `GENRES`, `SCALES`, `NOTE_SECONDS`, the composer and the synthesiser); change `temperature`; use three previous notes.
* *Viva.*
  1. *How is music "generated" autoregressively?* Each note is sampled from a distribution conditioned on the previous notes.
  2. *How does genre conditioning work?* A genre one-hot vector is an extra input to the network.
  3. *What is the spectral centroid?* The centre of mass of the spectrum — a measure of brightness.
  4. *What does the crest factor tell you?* Peak-to-RMS ratio: low for heavily compressed/distorted sound, high for dynamic sound.
  5. *How does Jukebox differ?* It compresses raw audio with a hierarchical VQ-VAE and models the codes with huge Transformers.
</details>

<details>
<summary><b>Experiment 7 — CycleGAN (<code>cyclegan.py</code>)</b></summary>

**What it does.** Learns to translate images between two domains **without paired examples**. Here domain A = a bright
square on a dark background and domain B = a dark square on a bright background (the two sets are drawn independently).

**Components.** `make_domains`; `CycleGAN` with generators `g_ab` (A→B) and `g_ba` (B→A), discriminators `d_a`, `d_b`;
`train_step` (generator update, then discriminator update); `translate_a_to_b`, `translate_b_to_a`, `cycle_a`, `cycle_b`;
`train_cyclegan`. The networks are tiny convolution networks built from the `Conv2D` layer in `_nn.py`.

**Maths.**
Adversarial loss (least squares): the generator minimises `(D(G(x)) − 1)²`; the discriminator minimises `(D(real) − 1)² + D(fake)²`.
Cycle-consistency loss: `|G_BA(G_AB(a)) − a| + |G_AB(G_BA(b)) − b|` (L1).
Identity loss (optional, `lambda_identity > 0`): `|G_AB(b) − b| + |G_BA(a) − a|` — a domain-B image given to the generator that
already outputs domain B should come out unchanged, which discourages needless colour changes.
Total generator loss: `adversarial + λ_cycle · cycle (+ λ_cycle · λ_identity · identity)`; `λ_cycle = 10` as in the document.

**Training.** For each batch: translate both ways, compute the three losses, back-propagate by hand, update the generators,
then update the discriminators on real vs freshly generated images (Adam, β₁ = 0.5). The batches of A and B are shuffled
independently, so there are no pairs.

**Run.** `dlspit genai cyclegan` or `run_experiment(epochs=60)` (≈ 6 s).

**Expected output.** Loss curves, then pictures *real A | translated to B | cycled back to A*, plus two numbers measured on
unseen images: the translation error against the known correct answer (the inverted image; `0` = perfect) and the cycle
error. **Result varies with the random seed** — see the limits below.

**Parameters.** `epochs`, `n_samples`, `image_size`, `lambda_cycle`, `lambda_identity`, `seed`. The document's `lr = 2e-4`
(for 256 × 256 photos and 100+ epochs) is replaced by `lr = 2e-3` in `train_cyclegan(lr=...)` because the problem is tiny.

**Limits.** 8×8 images, three convolution layers, no instance normalisation, no image pool. GAN training of this kind is
seed-sensitive: for some seeds the model inverts the background but not the small square, so the translation error stays near 0.3
instead of ≈ 0.02 (the cycle error is low in both cases, because cycle consistency alone cannot tell the two solutions apart).
Try `--seed` values and look at the printed pictures. The identity loss did not help in this particular toy setting.
The discriminator scores the whole image (the paper's PatchGAN scores patches, which cannot notice a missing object on an 8×8 image).

**Exam notes**
* *Summary.* Two GANs (A→B and B→A) tied together by a cycle-consistency loss so that unpaired data is enough.
* *Code walk-through.* `make_domains` → `CycleGAN.__init__` (the four networks) → `_one_direction` (adversarial, cycle, identity for one direction) → `_discriminator_loss` → `train_step` → `train_cyclegan` → evaluation in `run_experiment`.
* *Key functions.* `_one_direction`, `train_step`.
* *Likely modifications.* change `lambda_cycle` (try 1, 5, 10) and observe; turn on `lambda_identity=0.5`; swap L1 for L2 in the cycle loss; different domains in `make_domains`; change `hidden_dim`.
* *Viva.*
  1. *Why do we need cycle consistency?* With unpaired data, adversarial loss alone can map an input to *any* plausible target-domain image; the cycle loss forces the translation to keep the content so it can be undone.
  2. *Why two generators and two discriminators?* One pair per direction; both directions are needed for the cycle.
  3. *Role of the identity loss?* Preserves colour/tint and stabilises training when the input is already in the target domain.
  4. *Why least-squares GAN loss?* More stable gradients than the log loss; used in the original CycleGAN.
  5. *Why does the cycle error not prove the translation is right?* The generators can hide information in the translated image; you must also look at the translation itself.
</details>

<details>
<summary><b>Experiment 8 — Chatbot (<code>chatbot.py</code>)</b></summary>

**What it does.** A small neural network classifies the user's message into an *intent* and the bot answers with a pre-written
reply for that intent. **It is a retrieval bot, not a generative LLM** (the lab PDF uses DistilGPT2, which is not allowed).

**Components.** `INTENTS` (the local dataset: example sentences + answers), `tokenize`, `Chatbot.vectorize` (bag of words),
`Chatbot.train` (softmax regression with one hidden layer), `Chatbot.predict` / `respond` (confidence threshold and the
`out_of_scope` class for unknown input), `run_chatbot` (terminal loop).

**Maths.** `p = softmax(W₂·ReLU(W₁·x + b₁) + b₂)`; loss = cross-entropy; the bot answers with the arg-max intent only if its probability ≥ `threshold` (0.6)
and it is not `out_of_scope`.

**Run.** `dlspit genai chatbot` (interactive, `exit` to quit) or `dlspit genai chatbot --demo` (scripted conversation).

**Expected output.** 12 intents, ≈ 75 training sentences, loss ≈ 0.002 and 100 % training accuracy, then a sample chat in which
"what is the capital of france" is answered with "Sorry, I did not understand that…".

**Parameters.** `epochs`, `seed`; `threshold` and `hidden_dim` in `Chatbot(...)`; add intents by editing `INTENTS`.

**Limits.** Knows only its 11 topics, ignores word order, cannot generate new sentences, and can still be fooled by sentences that share words with a topic.

**Exam notes**
* *Summary.* Words → bag-of-words vector → neural classifier → intent → canned response, with a fallback for low confidence.
* *Code walk-through.* `INTENTS` → `vectorize` → `training_data` → `train` → `predict` → `respond` → `run_chatbot`.
* *Key functions.* `predict` (the unknown-input logic), `run_chatbot`.
* *Likely modifications.* add an intent (patterns + responses); change `threshold`; remove stop words; print the confidence; add a "name" memory.
* *Viva.*
  1. *Retrieval vs generative chatbot?* Retrieval picks an existing answer; generative models produce new text token by token.
  2. *Why a confidence threshold?* To avoid answering confidently when the question is outside the known topics.
  3. *Why an `out_of_scope` class?* Without examples of what *not* to answer, shared words like "what is the" trigger wrong intents.
  4. *Weakness of bag-of-words?* Ignores order and meaning ("dog bites man" = "man bites dog").
  5. *How would you make it generative?* Train a language model such as `mini_gpt.py` on dialogue data and sample replies.
</details>

<details>
<summary><b>Experiment 9 — Diffusion model, DDPM (<code>diffusion.py</code>)</b></summary>

**What it does.** Learns the "two moons" 2-D distribution by learning to remove noise, then generates new points starting from pure noise.

**Components.** `make_moons`, `make_schedule` (linear / cosine → β, α, ᾱ), `q_sample` (forward noising), `create_model` (MLP that
predicts the noise from `x_t` and `t/T`), `train_step` / `train_diffusion`, `sample` (reverse process), `chamfer_distance`,
`compare_schedules` (Part 2), `guidance_demo` (Part 3 substitute).

**Maths.** Forward: `x_t = √ᾱ_t·x₀ + √(1−ᾱ_t)·ε`. Training loss: `‖ε − ε_θ(x_t, t)‖²` at a random t.
Reverse step: `x_{t−1} = (x_t − β_t/√(1−ᾱ_t)·ε_θ)/√α_t + √β_t·z` (no `z` at the last step). Classifier-free guidance:
`ε = ε_uncond + w·(ε_cond − ε_uncond)`.

**Run.** `dlspit genai diffusion` or `run_experiment(epochs=300, T=50, schedule_kind="linear")` (≈ 10 s with the study and guidance parts).

**Expected output.** Real moons and generated moons in ASCII (the generated ones are recognisable but noisier); a table of
signal level √ᾱ_T for a few diffusion steps; Chamfer distance ≈ 0.09 (unrelated Gaussian noise ≈ 0.18, a second set of real points ≈ 0.03);
Part 2 table comparing linear/cosine and T; guidance demo where nearly all samples land on the requested moon.
In the Part 2 table a **small T with the linear schedule leaves a lot of signal at the last step** (√ᾱ_T ≈ 0.36 for T = 10) and gives the worst samples.

**Parameters.** `epochs`, `n_samples`, `T`, `schedule_kind` (`"linear"`/`"cosine"`), `study`, `guidance`, `seed`.

**Limits.** 2-D data; MLP instead of a U-Net; the toy guidance demo does not create "AI art" — the real text-to-image Stable Diffusion
of Part 3 is *not* included because it needs pretrained weights. Sample quality is modest.

**Exam notes**
* *Summary.* Add noise gradually (forward, fixed); train a network to predict that noise; generate by starting from noise and removing predicted noise step by step.
* *Code walk-through.* `make_moons` → `make_schedule` → `q_sample` → `train_step` → `train_diffusion` → `sample` → `chamfer_distance`.
* *Key functions.* `q_sample`, `train_step`, `sample`.
* *Likely modifications.* change `T`; switch to cosine; use another dataset (a circle); change the network width; print `x_t` at a few steps; plot the loss.
* *Viva.*
  1. *What does the network predict?* The noise ε that was added to produce `x_t`.
  2. *Why can we jump to any t in one step?* Sums of Gaussian noises are Gaussian, so `x_t` has a closed form in terms of `ᾱ_t`.
  3. *Linear vs cosine schedule?* Cosine destroys the signal more slowly at first and wastes fewer steps near pure noise.
  4. *Why must `ᾱ_T` be ≈ 0?* Sampling starts from pure noise, so the forward process must end there.
  5. *What is the guidance scale?* How strongly the sample is pushed towards the condition; larger = closer to the prompt, less diverse.
  6. *How is this linked to Stable Diffusion?* Same objective, but on image latents with a U-Net and a text encoder providing the condition.
</details>

---

# Explainable AI experiments (`dlspit xai`) — new in v2.2.0

> **Which version is this section about?** `dlspit xai <name>` now shows the **standalone script** (scikit-learn, and PyTorch for Grad-CAM
> and XRAI; see [show → copy → run](#genai-and-xai-show--copy--run)). This section describes the original **NumPy-only library version**
> (`--library`, Python API `dl_experiments.xai.<module>`) with its viva questions and exam modifications, which apply to both.
> The standalone scripts use scikit-learn models and `torch` where this section says "NumPy only", and they have no `plot=` argument
> (their figures are always drawn).

Ten Explainable AI (XAI) experiments, implemented **from scratch in Python + NumPy** (library version) for college practical examinations.
Every explanation method (Shapley values, PDP, ICE, LIME, Grad-CAM, Integrated Gradients / XRAI-style regions,
Gradient × Input, LRP, …) is written out in the module itself, and each one is applied to a small model that is
**trained here on synthetic data**.

* **No pretrained models** and **no dataset downloads** — nothing is fetched from Kaggle, scikit-learn, TensorFlow Hub, etc.; no internet is needed.
* **No XAI libraries** — the `shap`, `lime`, `captum`, … packages are *not* used (a test checks the imports). The library version does not use scikit-learn or scipy either; the standalone scripts do use scikit-learn / scipy for ordinary model training and the t-distribution.
* **NumPy only (library version)** — plots are optional (`pip install "dlspit[xai]"` adds Matplotlib). The neural networks reuse the small NumPy toolkit of the GenAI package (`dl_experiments/genai/_nn.py`).
* **Short, commented, modifiable code** — one file per experiment: data → model → explanation → demonstration.

> **Honest scope.** These are *educational* implementations on toy models. They show how each algorithm works and are
> checked against the numbers in the lab manuals; they are not drop-in replacements for the `shap`, `lime` or Captum libraries.
> Simplified versions say so in their name and docstring (for example **XRAI-style fixed regions**).

## A. Installation

```bash
pip install dlspit            # enough for the --library versions and for showing any experiment
pip install "dlspit[xai]"     # to RUN the standalone scripts: scikit-learn, scipy, matplotlib (+ pip install torch for grad-cam / xrai)
pip install "dlspit[all]"     # everything (TensorFlow, scikit-learn, pandas, matplotlib, ...)
```

Without Matplotlib every experiment still prints its results; asking for `--plot` then gives a clear message
(`Plotting needs matplotlib. Install it with: pip install "dlspit[xai]"`). Grad-CAM and XRAI can also write small PNG files without Matplotlib
(`--save-dir out`).

## B. The experiments

| # | Experiment | Python module | Main algorithm | Dependencies | CLI command |
|---|------------|---------------|----------------|--------------|-------------|
| 1 | Linear & logistic regression | `dl_experiments.xai.regression` | OLS, R², adj. R², t-statistics, 95 % CI, odds ratios | NumPy | `dlspit xai regression` |
| 2 | Decision tree & KNN | `dl_experiments.xai.decision_tree_knn` | CART (Gini), IF-THEN rules, impurity & permutation importance, KNN neighbours | NumPy | `dlspit xai tree-knn` |
| 3 | Shapley values (SHAP idea) | `dl_experiments.xai.shap` | exact Shapley formula + permutation sampling | NumPy | `dlspit xai shap` |
| 4 | Partial Dependence Plots | `dl_experiments.xai.pdp` | average prediction over a feature grid, PDP importance | NumPy | `dlspit xai pdp` |
| 5 | ICE curves | `dl_experiments.xai.ice` | one prediction curve per observation | NumPy | `dlspit xai ice` |
| 6 | XRAI-style regions | `dl_experiments.xai.xrai` | Integrated Gradients → fixed-grid region scores → ranking | NumPy | `dlspit xai xrai` |
| 7 | Grad-CAM | `dl_experiments.xai.grad_cam` | gradient-weighted feature maps + ReLU | NumPy | `dlspit xai grad-cam` |
| 8 | LIME | `dl_experiments.xai.lime` | perturb → kernel weights → weighted least squares | NumPy | `dlspit xai lime` |
| 9 | CBOW + Gradient × Input | `dl_experiments.xai.gradient_input` | CBOW embeddings; `A_i = x_i · ∂F/∂x_i` | NumPy | `dlspit xai gradient-input` |
| 10 | Layer-wise Relevance Propagation | `dl_experiments.xai.lrp` | LRP-ε and LRP-z⁺ rules, relevance conservation | NumPy | `dlspit xai lrp` |

Optional for all ten: `matplotlib` (extra `xai`) for `--plot`. No experiment needs TensorFlow, scikit-learn or scipy.

### What was implemented, and where it differs from the lab manuals

The numbering follows the PDFs in `XAI/expt_details/`. The completed notebooks/PDFs in `XAI/expt_done/` were read and checked, not copied.

| # | Manual / completed notebook | This package |
|---|----------------------------|--------------|
| 1 | Numbers are "given"; notebook downloads a Kaggle housing file and uses scipy | The manual's numbers **and** a model fitted here on synthetic data; the t-quantile is computed without scipy (table for df ≤ 30, series expansion above; max error < 5·10⁻⁴ vs scipy) |
| 2 | scikit-learn's Iris data and scikit-learn models | Synthetic "Iris-like" data; the tree (CART) and KNN are written from scratch |
| 3 | Permutation-sampling Shapley, plain Python, no NumPy | Both the **exact** Shapley formula and the permutation-sampling method (the manual's worked examples +3 and −9 are reproduced); uses NumPy for speed |
| 4 | PDP importance with std (K−1) and range/4 | As specified, plus the manual's 5 test cases |
| 5 | ICE of a 3-row linear model | The same table, plus a trained model with an interaction where the curves are *not* parallel |
| 6 | XRAI with **fixed regions** on given attributions; notebook needs an uploaded photo | Integrated Gradients on a small CNN trained here + the manual's fixed-grid region scoring. **Named "XRAI-style": it is not the full XRAI segmentation-and-merging algorithm** |
| 7 | Grad-CAM on a CNN (no completed notebook exists — the `XAI_Expt_7.pdf` in `expt_done` is a copy of the XRAI one) | A CNN trained from scratch on synthetic shapes; Grad-CAM written out step by step |
| 8 | LIME with weighted least squares (2-feature example) | The manual's example **and** a continuous-feature LIME on a trained black box. The manual's printed intermediate matrices contain arithmetic slips (see the note under Experiment 8) |
| 9 | CBOW embeddings + Gradient × Input on a text sentiment neuron, no NumPy | Both parts (with NumPy), plus Gradient × Input on a trained tabular network |
| 10 | LRP-ε on a 2-2-1 network, no libraries | The manual's network and its 5 test cases, plus a general LRP-ε / z⁺ for any ReLU network trained here |

## C. Usage

### Command line

```bash
dlspit xai --list                               # the ten experiments
dlspit xai shap                                 # SHOW the standalone script (theory + code)
dlspit xai shap --run                           # run that script
dlspit xai shap --library --instance 5          # NumPy-only library version; options need --library
dlspit xai pdp --library --feature 1 --epochs 200 --plot      # choose the feature; also draw a figure
dlspit xai tree-knn --library --k 7 --instance 3
dlspit xai grad-cam --library --target 0        # explain class 0 only
dlspit xai lrp --library --target 0 --instance 4
dlspit xai grad-cam --library --save-dir out    # also write PNG files into ./out
dlspit xai lrp --library --source               # print the LIBRARY source code
dlspit xai ice xrai                             # several experiments in a row (shown)
python -m dl_experiments xai lime               # same thing without the console script
```

Names are case-insensitive and accept aliases (`knn`, `gradcam`, `gxi`, `linear`, `shapley`, …) or the experiment number `1`–`10`.
Command-line options: `--epochs`, `--samples` (size of the synthetic data set), `--seed`, `--save-dir`, `--feature` (pdp, ice),
`--target` (grad-cam, xrai, gradient-input, lrp), `--instance` (shap, lime, gradient-input, lrp, tree-knn), `--k` (tree-knn), `--plot`, `--source`.
These options apply to the library version and therefore require `--library`. An option that an experiment does not have (for example
`--epochs` for `regression`) produces a clear error. Parameters such as `epsilon`, `rule`, `kernel_width`, `grid`, `steps`, `n_grid`,
`n_permutations` are Python arguments of the library (see below); in the standalone scripts they are constants at the top of the file.

### Python

Every module has `run_experiment(...)` with working defaults (prints a report, returns a dictionary), plus small functions you can call directly:

```python
from dl_experiments.xai.regression import evaluate_linear_regression, interpret_logistic_regression, run_experiment
r = evaluate_linear_regression(y_actual=[3, 5, 7, 9, 12], y_pred=[3.2, 4.8, 7.1, 9.3, 11.6],
                               coefs=[1.9], se_coefs=[0.15], feature_names=["hours"])
print(r["r2"], r["adj_r2"], r["t_statistics"], r["ci_lower"], r["ci_upper"])
print(interpret_logistic_regression(["age", "income"], [0.06, 0.85])["rows"][1]["interpretation"])
```

```python
from dl_experiments.xai.decision_tree_knn import DecisionTree, knn_explain, make_iris_like
X, y, features, classes = make_iris_like(150)
tree = DecisionTree(max_depth=3, feature_names=features, class_names=classes).fit(X, y)
print(tree.rules()); print(tree.explain(X[0])["steps"])
print(knn_explain(X, y, X[0], k=5)["neighbors"])
```

```python
import numpy as np
from dl_experiments.xai.shap import explain_prediction
model = lambda A: 2 * A[:, 0] + 3 * A[:, 1] + 5 * A[:, 2]            # any function of a 2-D array works
e = explain_prediction(model, np.array([4., 2., 3.]), np.array([[5., 1., 4.], [2., 5., 1.]]), ["X1", "X2", "X3"])
print(e["baseline"], e["phi"], e["prediction"])                       # baseline + sum(phi) == prediction
```

```python
import numpy as np
from dl_experiments.xai.pdp import calculate_pdp, pdp_importance
from dl_experiments.xai.ice import calculate_ice
X = np.random.default_rng(0).normal(size=(100, 2))
f = lambda A: A[:, 0] ** 2 + A[:, 1]
grid, pdp_values = calculate_pdp(f, X, feature=0, n_grid=10)
grid, curves = calculate_ice(f, X, feature=0, n_grid=10)               # shape (100, 10): one curve per row
print(pdp_importance(pdp_values), np.allclose(curves.mean(axis=0), pdp_values))
```

```python
import numpy as np
from dl_experiments.xai.lime import explain_instance
f = lambda A: 1 / (1 + np.exp(-(A[:, 0] - A[:, 1])))                    # a black box
e = explain_instance(f, np.array([1.0, 0.5]), scale=np.array([1.0, 1.0]), n_samples=500, kernel_width=1.0)
print(e["coefficients"], e["weighted_r2"])
```

```python
from dl_experiments.xai.grad_cam import SmallCNN, make_dataset, train_cnn, grad_cam, two_object_image
images, labels = make_dataset(300)
cnn = SmallCNN(); train_cnn(cnn, images, labels, epochs=15)
r = grad_cam(cnn, two_object_image(), target=1)                        # target 1 = "vertical bar"
print(r["activations"].shape, r["alpha"].shape, r["heatmap"].shape, r["heatmap_resized"].shape)
```

```python
from dl_experiments.xai.xrai import integrated_gradients, grid_regions, region_scores, rank_regions
attribution, gap = integrated_gradients(cnn, two_object_image(), target=1, steps=30)   # cnn from the previous snippet
regions = grid_regions(attribution.shape, 3, 3)
print(rank_regions(region_scores(attribution, regions))[:3])           # most important regions first
```

```python
from dl_experiments.xai.gradient_input import sentiment_gradient_x_input, SENTIMENT_WEIGHTS
print(sentiment_gradient_x_input([1, 1, 1, 1, 0], SENTIMENT_WEIGHTS)["attribution"])   # "the movie was good"
```

```python
import numpy as np
from dl_experiments.xai.lrp import lab_manual_network, propagate_relevance
weights, biases = lab_manual_network()
r = propagate_relevance(weights, biases, np.array([1.0, 1.0]), target=0, epsilon=1e-6)
print(r["input_relevance"])                                            # ~[1. 2.]  (exactly [1, 2] when epsilon=0; sum ~ output logit = 3)
```

The registry gives access by name as well: `from dl_experiments.xai import list_experiments, run_experiment, show_source`.

### Notes

* **Everything is reproducible**: each experiment takes `seed`; the same seed gives identical output.
* **Speed**: every experiment runs in well under a second at its defaults on an ordinary laptop (`pytest -q` for the whole package takes ≈ 6 s).
* **Tests**: `pip install pytest && pytest -q` from the repository root. They reproduce the numbers printed in the lab manuals, compare hand-written gradients with
  finite differences, check the Shapley efficiency property, the PDP = mean(ICE) identity, relevance conservation, and the CLI; no network is used.
* **Models are tiny on purpose.** An explanation says something about *the toy model it was computed on*, not about the real world.

## D. Experiment guide: algorithm, implementation, exam preparation

Click an experiment to expand it. For each one: objective, algorithm and formulas, how the code is organised, how to run it, what you will
see, which parameters to change, limitations, likely modifications and viva questions. All numbers quoted are from the default run (`seed=0`).

<details>
<summary><b>Experiment 1 — Linear &amp; logistic regression: evaluation and coefficient interpretation (<code>regression.py</code>)</b></summary>

**Objective.** Evaluate a regression model with R², adjusted R², t-statistics and 95 % confidence intervals, and interpret logistic-regression coefficients as odds ratios.

**Formulas.** `SSE = Σ(y−ŷ)²`, `SST = Σ(y−ȳ)²`, `R² = 1 − SSE/SST`, `adj.R² = 1 − (1−R²)(n−1)/(n−k−1)`, `t = b/SE(b)`,
`95 % CI = b ± t_crit(df)·SE(b)` with `df = n−k−1`; logistic: `odds ratio = e^b`, `% change in odds = (e^b − 1)·100`, probability `σ(b₀ + b·x)`.

**Implementation.** *Fitting* (`fit_linear_regression` by the normal equations, `fit_logistic_regression` by Newton/IRLS) is separate from *interpretation*
(`evaluate_linear_regression`, `interpret_logistic_regression`), which also accept numbers you type in, as in the manual. `t_critical` needs no scipy.

**Run.** `dlspit xai regression` · `run_experiment(n_samples=100, n_features=3)`.

**Expected output.** (1a) the manual's numbers: R² = 0.9771, adjusted R² = 0.9084, t = 12.24 / 3.43 / −3.33 and — because n = 5, df = 1 — every 95 % interval contains 0
(not significant); (1b) a model fitted on synthetic data (true coefficients 2, −1.5, 0.5) with R² ≈ 0.90 and significant intervals; (2) odds ratios 1.06 / 2.34 / 0.67 / 3.49 with the ranking Credit Score > Income > Loan Amount > Age,
and a logistic model fitted on synthetic data.

**Parameters.** `n_samples`, `n_features`, `seed`; `noise` in `make_regression_data`. To interpret your *own* model call `evaluate_linear_regression(y, y_pred, coefs, se_coefs, names)`.

**Limitations.** The 95 % level is fixed; coefficients are associations, not causal effects; coefficient size is comparable only when the features are on the same scale.

**Likely modifications.** change `n_features` (a 4th feature has true coefficient 0 and a small t-statistic); change `noise` or `n_samples` and watch R² and the intervals; feed your own `y`, `ŷ`, coefficients and standard errors; rank logistic features by `|coef|`.

**Viva.** (1) *What does R² mean?* The share of the variation of y explained by the model. (2) *Why adjusted R²?* R² never decreases when a predictor is added; the adjustment penalises useless predictors. (3) *What does a CI containing 0 mean?* The coefficient is not significantly different from 0 at 5 % (|t| < t_crit). (4) *What is an odds ratio of 3.49?* A one-unit increase multiplies the odds of the positive class by 3.49 (+249 %). (5) *Why is a negative logistic coefficient meaningful?* It lowers the log-odds, hence the probability, of the positive class.
</details>

<details>
<summary><b>Experiment 2 — Decision trees and K-Nearest Neighbours (<code>decision_tree_knn.py</code>)</b></summary>

**Objective.** Explain a decision tree (visualisation, rules, importances) and a KNN classifier/regressor.

**Formulas.** Gini `G = 1 − Σ p_c²`; split cost `(n_L·G_L + n_R·G_R)/n`; impurity importance of a feature = total (weighted) impurity decrease of its splits; permutation importance = accuracy drop after shuffling the column; KNN distance `√Σ(xⱼ−zⱼ)²` on standardised features, vote = majority, regression = mean of the neighbours.

**Implementation.** `DecisionTree` (CART, nodes are dictionaries) with `fit`, `predict`, `explain` (decision path of one row), `rules`, `to_text`, `importances`; `permutation_importance`; `knn_explain` (neighbours, distances, votes, tie-break) and `knn_predict`. Data: `make_iris_like` (synthetic).

**Run.** `dlspit xai tree-knn` · `run_experiment(max_depth=3, k=5, instance=0)`.

**Expected output.** A text tree with root `petal_length <= 2.80`, 3 leaves and 3 IF-THEN rules, impurity importance 0.5 / 0.5 for the two petal features, permutation importance 0.42 / 0.33, test accuracy tree 100 % and KNN 97.8 %, the decision path and the 5 nearest neighbours for one flower (test row 0 is a case where the tree is right and KNN votes 4:1 for the wrong class), and a KNN-regression example with the manual average.
(Two features separate setosa equally well; the tie decides the root, scikit-learn may pick the other one — accuracy is the same.)

**Parameters.** `max_depth`, `k`, `instance`, `n_samples`, `seed`; in `knn_explain`: `normalize`, `task`.

**Limitations.** Greedy splits; no pruning besides `max_depth`/`min_samples_split`; KNN depends on distance and scaling and keeps all training data.

**Likely modifications.** change `max_depth` (1 → under-fitting, 6 → more rules); change `k` (1 = follows noise, large = over-smooths); explain another row with `--instance`; turn off scaling (`normalize=False`) and see the neighbours change; replace `gini` by entropy (a small code change in `gini`).

**Viva.** (1) *Why are trees interpretable?* Each prediction follows a root-to-leaf path that is a readable rule. (2) *Gini vs entropy?* Both measure impurity; Gini = 1 − Σp², entropy = −Σp log p; results are usually similar. (3) *Impurity vs permutation importance?* Impurity is computed on training data during fitting and can favour many-valued features; permutation is computed afterwards on held-out data and reflects the effect on accuracy. (4) *Why standardise for KNN?* Otherwise features with large units dominate the distance. (5) *Which is more explainable?* A tree has a global rule set; KNN is only locally explainable ("these neighbours voted").
</details>

<details>
<summary><b>Experiment 3 — Shapley values: the idea behind SHAP (<code>shap.py</code>)</b></summary>

**Objective.** Explain one prediction by splitting `prediction − baseline` fairly among the features.

**Formula.** `φᵢ = Σ_{S ⊆ N∖{i}} [ |S|!·(M−|S|−1)! / M! ] · [ v(S∪{i}) − v(S) ]`, where `M` = number of features; `v(S)` = average prediction over a background set when the features in `S` take the explained row's values and the rest come from a background row; the weight is the probability that exactly `S` precedes `i` in a random ordering; the bracket is the marginal contribution. Efficiency: `baseline + Σφᵢ = f(x)`.

**Implementation.** `exact_shapley_values` (all `2^M` coalitions, vectorised, `M ≤ 12`); `permutation_shapley_values` (the manual's random-order sampling, with standard errors); `before_after_vectors` (the manual's before/after vectors); `explain_prediction` (both methods, returns baseline, φ, prediction, efficiency gap). **Not** the `shap` library; masking assumption = "replace by background data".

**Run.** `dlspit xai shap --instance 3` · `run_experiment(instance=3, n_permutations=2000)`.

**Expected output.** The manual's examples (+3 and −9), exact φ for `f = 2x₁+3x₂+5x₃`; then for a trained loan model: baseline 0.548, φ(income) −0.15, φ(debt) −0.21, φ(credit_score) +0.12 (sum −0.23) giving 0.316 exactly (gap ≈ 1e-16), sampled φ close to the exact ones, and a comparison with permutation (global) importance.

**Parameters.** `instance`, `n_permutations`, `n_samples`, `epochs`, `seed`; your own `predict_fn` and `background` in `explain_prediction`; `method="exact"|"permutation"`.

**Limitations.** Exponential cost; results depend on the background; mixing rows creates unrealistic points for correlated features; it explains the model, not the world.

**Likely modifications.** explain another row (`--instance`); change `n_permutations` (10 → 10 000) and watch the error shrink; use your own model function; add a 4th feature to the worked example (a permutation then has 4 entries); compare two seeds.

**Viva.** (1) *What is a Shapley value?* A feature's average marginal contribution over all coalitions/orderings. (2) *Why can φ be negative?* The feature pushed this prediction below the baseline. (3) *What is the baseline?* The average prediction on the background data. (4) *Why approximate?* Exact enumeration costs 2^M model evaluations; sampling random orders estimates the mean. (5) *SHAP vs ordinary feature importance?* Importance is global and unsigned; Shapley values are local, signed and add up to the prediction.
</details>

<details>
<summary><b>Experiment 4 — Partial Dependence Plots (<code>pdp.py</code>)</b></summary>

**Objective.** Show the *average* effect of a feature on the model and rank features by the variability of their PDP.

**Formulas.** `PD(v) = (1/n) Σᵢ f(xᵢ with feature s := v)`; importance: numerical `I = √( Σ(PD−mean)²/(K−1) )`, categorical `I = (max PD − min PD)/4` (Greenwell et al. 2018, as required by the manual). Cost `O(n·K)` predictions.

**Implementation.** `calculate_pdp(predict_fn, X, feature, grid=None, n_grid=20)` overwrites one column, predicts, averages. `pdp_importance` implements both formulas (and refuses `K = 1`). Data: synthetic machines (temperature — non-linear, vibration — linear, type A/B/C — categorical) and a neural-network black box.

**Run.** `dlspit xai pdp --feature 1` · `run_experiment(feature=0, n_grid=12)`.

**Expected output.** The manual's checks (0.2219, 0.1125, 0), then ASCII bar PDPs for temperature (U-shaped, I ≈ 1.75), vibration (rising line, I ≈ 0.94) and machine type (B highest, I ≈ 0.50), and the importance ranking.

**Parameters.** `feature`, `n_grid`, `n_samples`, `epochs`, `seed`; `grid=` for explicit values (categorical codes).

**Limitations.** Averages hide individual differences (see ICE); strongly correlated features give unrealistic rows and extrapolation; describes the model, not causation; numerical and categorical importances are not strictly comparable.

**Likely modifications.** another feature (`--feature 1`); finer grid (`--n_grid` via Python); compute the PDP of your own `predict_fn`; manually compute `I` for one curve; compare the PDP with the ICE curves (Experiment 5).

**Viva.** (1) *What does a PDP show?* The model's average prediction as one feature varies and the others keep their observed values. (2) *Why `K−1`?* It is the sample standard deviation. (3) *Main limitation?* Unrealistic combinations for correlated features and hidden heterogeneity. (4) *Complexity?* O(n·K) predictions. (5) *Is a PDP causal?* No, it describes the model.
</details>

<details>
<summary><b>Experiment 5 — Individual Conditional Expectation (<code>ice.py</code>)</b></summary>

**Objective.** Show how the prediction changes for *each* observation when one feature varies.

**Formula.** `ICEᵢ(v) = f(xᵢ with feature s := v)`; `PDP(v) = mean_i ICEᵢ(v)`.

**Implementation.** `calculate_ice` returns `(grid, ice)` with `ice.shape == (n_observations, n_grid)`; `average_ice_curve` (= PDP); `center_ice` (c-ICE, curves start at 0).

**Run.** `dlspit xai ice --feature 1` · `run_experiment(feature=0, n_curves=8)`.

**Expected output.** The manual's table for `Ŷ = 3.5 + 3X₁ + 4X₂` (26.5 / 29.5 / 32.5, …, parallel, slope 3, PDP = 30.5 / 33.5 / 36.5), then a neural network trained on `y = x₁·x₂`: slopes from about −1.9 to +1.7 (not parallel) while the PDP is almost flat (range ≈ 0.06), and a check that PDP = mean of the ICE curves.

**Parameters.** `feature`, `n_grid`, `n_curves`, `n_samples`, `epochs`, `seed`.

**Limitations.** Too many curves are unreadable (use a sample); unrealistic combinations for correlated features; model behaviour, not causation.

**Likely modifications.** vary X₂ instead of X₁ (slope becomes 4); more or fewer curves; centred ICE; plot with `--plot`; use your own model function.

**Viva.** (1) *PDP vs ICE?* PDP is the average of the ICE curves; ICE shows each observation. (2) *Why parallel curves in the manual?* The model is additive, ∂Ŷ/∂X₁ = 3 for everyone. (3) *What do crossing/diverging curves mean?* An interaction or heterogeneous effect. (4) *Why can a flat PDP be misleading?* Opposite individual effects cancel in the average. (5) *Cost?* One prediction per observation per grid value.
</details>

<details>
<summary><b>Experiment 6 — XRAI-style region attribution (<code>xrai.py</code>)</b></summary>

**Objective.** Turn pixel attributions into a ranking of image *regions*. **This is the fixed-region educational version from the manual, not the full XRAI algorithm** (no segmentation, no greedy merging).

**Formulas.** Integrated Gradients `IGᵢ = (xᵢ − x′ᵢ)·∫₀¹ ∂F(x′+α(x−x′))/∂xᵢ dα` (midpoint rule, baseline `x′ = 0`); completeness `ΣIGᵢ ≈ F(x) − F(x′)`; region score `Score(Rⱼ) = (1/|Rⱼ|) Σ_{i∈Rⱼ} Aᵢ` (signed — negative regions argue against the class).

**Implementation.** `integrated_gradients(cnn, image, target, steps)` (returns the completeness gap), `grid_regions`, `region_scores(mode="mean"|"sum")`, `rank_regions`, `region_attribution_map`, `top_regions_mask`. The CNN is the one of Experiment 7, trained here on synthetic shapes.

**Run.** `dlspit xai xrai --target 1` · `run_experiment(grid=3, target=1, steps=30)`.

**Expected output.** The manual's 4×4 example (R4 = 0.80 > R1 = 0.20 > R2 = R3 = 0.10), then for a two-object image and target "vertical bar": completeness gap ≈ 0.02 (sum 3.515 vs 3.537), the 3×3 region ranking with the vertical bar's region first and the horizontal bar's region last (negative evidence), and an erasure check.

**Parameters.** `grid` (n×n regions), `target`, `steps`, `epochs`, `n_samples`, `seed`; `mode="sum"` in `region_scores`; `baseline` in `integrated_gradients`.

**Limitations.** Fixed regions ignore object boundaries; mean scoring favours small dense regions; quality depends on the pixel attributions.

**Likely modifications.** change `grid` (2, 3, 4, 6); use `mode="sum"` and compare the ranking; change the target class; change `steps`; make one region negative and re-rank.

**Viva.** (1) *What is XRAI for?* Region-level (human-readable) explanations of pixel attributions. (2) *IG vs XRAI?* IG gives pixel scores; XRAI aggregates and ranks regions. (3) *Why average and not sum?* Regions of different size would otherwise be unfair; here regions are almost equal so both work. (4) *Why not take absolute values?* The sign tells whether a region supports or opposes the class. (5) *What was simplified?* Segmentation and region merging were replaced by a fixed grid.
</details>

<details>
<summary><b>Experiment 7 — Grad-CAM (<code>grad_cam.py</code>)</b></summary>

**Objective.** Show which image regions support a chosen class, using the gradients that flow into a convolutional layer.

**Formulas.** `αₖ = (1/Z) Σᵢⱼ ∂y_c/∂Aᵏᵢⱼ`, `L_c = ReLU(Σₖ αₖ Aᵏ)`, where `y_c` is the class score (logit), `Aᵏ` the k-th feature map and `Z` the number of positions; the map is scaled to [0, 1] and resized bilinearly.

**Implementation.** `SmallCNN`: Conv3×3(4) → ReLU → MaxPool → **Conv3×3(6) → ReLU (the selected layer: 6 maps of 6×6)** → Flatten → Dense(3). Trained from scratch on synthetic 12×12 images (horizontal bar / vertical bar / square). `grad_cam` returns `activations`, `gradients`, `alpha`, `raw_cam`, `heatmap`, `heatmap_resized`. Back-propagation is hand-written (checked against finite differences).

**Run.** `dlspit xai grad-cam --target 1` · `run_experiment(target=None)` explains all three classes.

**Expected output.** Test accuracy ≈ 99 %; for the two-object image (vertical bar left, horizontal bar bottom-right) the "vertical bar" heat map centres on the left object (col ≈ 1.6) and the "horizontal bar" map on the bottom-right object (col ≈ 7.6) — the heat map follows the chosen class; raw map peaks are small for wrong classes.

**Parameters.** `target`, `epochs`, `n_samples`, `seed`. For other pictures call `grad_cam(cnn, your_12x12_array, target)`; a different layer needs a change in `SmallCNN` (end `features` at another layer).

**Limitations.** Coarse (6×6) resolution; shows supporting evidence only (ReLU); needs a convolutional layer; the toy CNN knows only three shapes.

**Likely modifications.** another target class (`--target 0/1/2`); another image; more training epochs; print `alpha` and explain which map matters most; remove the ReLU and observe negative evidence.

**Viva.** (1) *Why gradients?* They say how much each feature map influences the class score. (2) *Why global average pooling of gradients?* One importance weight per map. (3) *Why ReLU?* Keeps only regions that increase the class score. (4) *Why is Grad-CAM class-discriminative?* `y_c` — and so `α` — changes with the target class. (5) *Grad-CAM vs Integrated Gradients?* Grad-CAM is coarse and layer-based; IG is pixel-level and needs a baseline.
</details>

<details>
<summary><b>Experiment 8 — LIME (<code>lime.py</code>)</b></summary>

**Objective.** Explain one prediction of a black box with a simple model that is only valid near that instance.

**Formulas.** perturb `zᵢ = x + s·N(0,1)`; distance `dᵢ = ‖(zᵢ−x)/s‖`; weight `wᵢ = exp(−dᵢ²/σ²)`; weighted least squares `β = (XᵀWX)⁻¹XᵀWy` with `X = [1, standardised offsets]`, `y = f(zᵢ)`; the coefficients are the local effects of +1 standard deviation. Default `σ = 0.75·√(number of features)`.

**Implementation.** `generate_perturbations`, `euclidean_distances`, `kernel_weights`, `weighted_least_squares`, `explain_instance` (returns coefficients, intercept, weighted R² "fidelity", effective sample size, and all perturbed data), `lab_manual_example`. **Not** the `lime` library.

**Run.** `dlspit xai lime --instance 3` · `run_experiment(n_perturbations=1000, kernel_width=None)`.

**Expected output.** The manual's example (weights 1, 0.368, 0.368, 0.135; `β = [0.307, 0.354, 0.254]`, so β₁ > β₂ > 0); for a trained loan model: P(approved) = 0.316, local coefficients income +0.27, credit_score +0.20, debt −0.14, age ≈ 0, fidelity R² ≈ 0.88; a *different* row gives different coefficients (local!); a wide kernel lowers the fidelity.

> **Note on the lab manual.** Section 15/16 of the manual prints `XᵀWX[1,2] = 0.368` and `XᵀWy = [1.600, 1.158, 1.079]`. Recomputing from its own tables gives
> `XᵀWX[1,2] = 1.000` and `XᵀWy = [1.405, 1.158, 1.121]` (e.g. 1·0.9 + 0.368·0.7 + 0.368·0.6 + 0.135·0.2 = 1.405). The conclusion β₁ > β₂ > 0 is unchanged. The tests assert the recomputed values.

**Parameters.** `instance`, `n_perturbations`, `kernel_width`, `n_samples`, `epochs`, `seed`; your own `predict_fn`, `scale` in `explain_instance`.

**Limitations.** Depends on the perturbation scale and kernel width; unstable for small samples; linear surrogate only (no sparsity/Lasso); no discretisation, text or image modes.

**Likely modifications.** change `n_perturbations` (50 vs 5000) and compare stability; change `kernel_width` (0.5, 1.5, 10); explain another instance; change the black-box epochs; repeat the manual example with `sigma=0.5`.

**Viva.** (1) *Why local?* One straight line cannot describe the model globally, but it can near x. (2) *Role of the kernel?* Samples close to x get larger weights. (3) *What does a coefficient mean?* The change of f for +1 std of that feature, near x. (4) *How to know if the explanation is trustworthy?* Check the weighted R² (fidelity). (5) *LIME vs SHAP?* LIME fits a local surrogate; Shapley values average marginal contributions and add up exactly.
</details>

<details>
<summary><b>Experiment 9 — CBOW embeddings and Gradient × Input (<code>gradient_input.py</code>)</b></summary>

**Objective.** (A) learn word embeddings with CBOW; (B/C) attribute a neural-network output to its inputs with Gradient × Input.

**Formulas.** CBOW: `h = mean(context embeddings)`, `z = h·W′`, softmax, loss `−log p(target)`, gradient `p − onehot`. Gradient × Input: `Aᵢ = xᵢ·∂F/∂xᵢ`; for `ŷ = σ(wᵀx+b)`, `∂ŷ/∂xᵢ = ŷ(1−ŷ)wᵢ`.

**Implementation.** Part A: `build_vocabulary`, `cbow_examples`, `train_cbow`, `cosine_similarity` (corpus "the movie was good/bad"). Part B: `sentiment_gradient_x_input` (the manual's weights `[0.1, 0.2, 0.1, 2.0, −2.0]`, typed in). Part C: `calculate_attributions(net, x, target, output="logit"|"probability")` on a network trained here, with hand-written back-propagation.

**Run.** `dlspit xai gradient-input --instance 2 --target 1` · `run_experiment(cbow_epochs=1000)`.

**Expected output.** CBOW loss 6.3 → 0.005 and 2-D embeddings (good/bad nearly parallel because they occur in identical contexts — the corpus is far too small for meaning); "the movie was good": z = 2.4, P = 0.9168, attributions 0.00763 / 0.01525 / 0.00763 / **0.1526** / 0; "the movie was bad": `bad` gets −0.28; a tabular example with gradients and attributions per feature; the saturation demo — when the output probability saturates, its attributions shrink towards 0 while the logit attributions stay large.

**Parameters.** `instance`, `target`, `epochs`, `cbow_epochs`, `seed`; `output` and your own network in `calculate_attributions`; any sentence vector in `sentiment_gradient_x_input`.

**Limitations.** Local slope only: saturation (vanishing gradients), dependence on the reference 0 and on input scaling/encoding; Σ attributions equals the output only for bias-free piecewise-linear networks (a test checks this case).

**Likely modifications.** explain another row or class; attribute the logit vs the probability; try the sentence "the movie was" or change a weight; change the embedding learning rate/epochs and compare cosine similarities.

**Viva.** (1) *What is Gradient × Input?* Sensitivity times the actual input value. (2) *Why is the attribution of an absent word 0 even with a large weight?* x = 0. (3) *Why use the logit?* The sigmoid/softmax gradient `ŷ(1−ŷ)` vanishes when saturated. (4) *What is CBOW?* Predict a word from the average embedding of its context. (5) *Relation to LRP?* For ReLU networks, LRP-ε equals Gradient × Input (tested in this package).
</details>

<details>
<summary><b>Experiment 10 — Layer-wise Relevance Propagation (<code>lrp.py</code>)</b></summary>

**Objective.** Redistribute the output score backwards, layer by layer, until every input feature holds a share of relevance.

**Formulas.** LRP-ε: `Rᵢ = Σⱼ [ aᵢ wᵢⱼ / (zⱼ + ε·sign(zⱼ)) ] Rⱼ` with `zⱼ = Σᵢ aᵢ wᵢⱼ + bⱼ`; LRP-z⁺ uses only positive weights and no bias. Output relevance = logit of the target output. Conservation: `ΣRᵢ = ΣRⱼ` exactly only if biases = 0 and ε = 0.

**Implementation.** `forward_pass` (ReLU hidden layers, linear output), `propagate_relevance(weights, biases, x, target, epsilon, rule)` returning all layer relevances and the per-layer conservation gaps, `weights_from_network`, `lab_manual_network`. A zero denominator (possible only for ε = 0) passes on relevance 0 instead of NaN.

**Run.** `dlspit xai lrp --instance 2 --target 1` · `run_experiment(epsilon=1e-6, rule="epsilon")`.

**Expected output.** The manual's table: for x = (1,1): h = (1,2), logit 3, p = 0.9526, `R_h = (1,2)`, `R_x = (1,2)`; the 4 other test cases (including (0,0), where the logit is 0 and all relevances are 0, finite); then a trained 4→8→2 network: input relevances, their sum versus the logit (the gap comes from biases and ε), the identical Gradient × Input values, and the z⁺ result.

**Parameters.** `epsilon`, `rule` (`"epsilon"` or `"z_plus"`), `target`, `instance`, `epochs`, `n_samples`, `seed`.

**Limitations.** Rule choice matters; ε-LRP on ReLU networks reduces to Gradient × Input (so it inherits its weaknesses); biases absorb relevance; negative relevance needs careful reading.

**Likely modifications.** change `epsilon` (0, 1e-6, 1, 100) and watch the conservation gap; switch to `rule="z_plus"`; explain the other class (`--target 0`); change the network weights in `lab_manual_network`; try x = (3, 0).

**Viva.** (1) *Why start at the output logit?* It is the score we want to explain; relevance is then conserved layer by layer. (2) *Role of ε?* Stabilises small/zero denominators; larger ε absorbs relevance and breaks conservation. (3) *Why is conservation only approximate?* Biases and ε take a share. (4) *LRP vs gradients?* LRP redistributes the *score*; gradients measure sensitivity. (5) *Can relevance be negative?* Yes — the input argued against the target.
</details>

---

## License

MIT