Metadata-Version: 2.4
Name: consennet
Version: 0.1.2
Summary: Cellular Consensus Network -- a cellular, message-passing architecture with cost independent of input size. Original concept and architecture by Ömür Bera Işık.
Author-email: Ömür Bera Işık <omurbera01@gmail.com>
License: Apache-2.0
Keywords: deep-learning,pytorch,neural-network,consensus,cellular
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.0
Requires-Dist: numpy
Provides-Extra: jax
Requires-Dist: jax; extra == "jax"
Requires-Dist: equinox; extra == "jax"
Provides-Extra: hf
Requires-Dist: transformers; extra == "hf"
Requires-Dist: accelerate; extra == "hf"
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"

# ConsenNet v0.1.2 — Cellular Consensus Network

**Original concept and architecture: Ömür Bera Işık** (see
`ConsenNet_Report` and `phase0/consennet_original.py`, the untouched
reference implementation). Everything labeled *implementer's addition* in the
source was built around that architecture: batched reimplementation,
experiments, benchmarks, backends, integrations, and documentation. Nothing
in the core idea is mine; every deviation from the original code is labeled
and justified in-file.

The architecture in one paragraph: *Cores* (state vectors) are grouped into
*Cells*. Within a cell, cores exchange gated messages and update through a
GRU. Cells communicate only through a compressed consensus *signal*
(`d_signal` ≪ core width). Each cell asynchronously decides when it is
confident enough to *close*; closed cells stay silent unless the consensus
drifts far enough from their last signal to *wake* them. Rounds run for a
fixed, per-task budget (usually 3-4 after tuning) — **not** because harder
inputs are detected and given more computation: six experiments
(`reports/ARCHITECTURE_FINDINGS.md`) disproved that "emergent depth" claim,
most decisively a 3-SAT test where accuracy fell 65 points across the
satisfiability phase transition while rounds used stayed flat to two decimal
places. Depth is a hyper-parameter here, not an adaptive mechanism.

## Status: v0.1.2

v0.1.1 fixed v0.1.0's 12 enumerated failures and trained every task to
convergence. v0.1.2 asked a harder question of the result — *why* does it
work, and against the right competitors? — found that granularity (many
simple cores) is the architecture's real, 63-sigma-confirmed advantage,
found that the efficiency claim shrinks against a fair (Perceiver-style)
baseline, and found that adaptive depth does not hold up. All three findings
are now folded into the package's defaults and documentation rather than
left as a separate report. Full evidence: `reports/ARCHITECTURE_FINDINGS.md`.

**Three of four v0.1.1 head-to-head verdicts still stand** (`reports/BENCHMARK_REPORT_v0.1.1.md`):

| task | v0.1.0 (250-600 steps) | v0.1.1 (trained) | winner |
|---|---|---|---|
| image acc (CN / TF) | 74.9 / 79.7% | **98.09 / 97.31%** | ConsenNet +0.78 pp (with round dropout) |
| audio acc (CN / TF) | 93.8 / 99.3% | **100.0 / 99.67%** | ConsenNet +0.33 pp |
| multimodal acc (cnmix / TF) | 42.0 / 48.4% | **88.42 / 78.83%** | ConsenNet +9.6 pp (mean) |
| text bits/char (CN / TF) | 4.71 / 4.78 | **2.93 / 2.66** | Transformer by 0.27 bits |

Image and audio results are converged (plateau-detected). Text and multimodal
were still improving at their budget ceilings — equal-budget comparisons, not
converged ones, and labeled as such throughout the report. Audio and
multimodal still rest on menu-matched (not exactly parameter-matched)
baselines whose misses favor the Transformer in every case where it lost —
the same correction that turned image from a "win" into a near-tie has not
yet been applied there; see `reports/ARCHITECTURE_FINDINGS.md` §5.

The image row uses an **exactly parameter-matched** baseline (`consennet.budget`,
a feature ported from physis-lm). The originally shipped baseline carried 6.4%
extra parameters and scored *worse* (96.30%); rebuilding it at exactly 47,598
parameters raised it to 97.31%.

## Layout

```
consennet/            the package (torch core, encoders, multimodal, data,
                      training, jax backend, hf integration, devpanel)
phase0/               original code (untouched) + replication experiments
tests/                148 tests (parity, modalities, jax, hf, devpanel, fixes, optim)
benchmarks/           bench.py harness, 28 result JSONs, analyze.py
scripts/              speed_opt.py, scale_sweep.py
reports/              EXPERIMENT_REPORT, SPEED_REPORT, BENCHMARK_REPORT,
                      SCALE_REPORT, CLOSING_SUMMARY, figures/, raw JSON
```

## Module status

| module | status | notes |
|---|---|---|
| `consennet.spec` | complete, tested | single validator for all entry points |
| `consennet.torch.model` | complete, tested | math parity vs the original; ragged execution (F5, bit-identical, cost ∝ rounds); negative conf-bias init (F1); median stopping for REINFORCE eval (F4); differentiable `soft_rounds` |
| `consennet.torch.encoders` | complete, tested | text/image/audio → core states; size-agnostic; spectrogram choice justified in-file |
| `consennet.torch.multimodal` | complete, tested | separate + mixed + hybrid routing (F2). Measured: mixed routing fuses (knock-out costs 39-57 pp); separate routing does not, even at 4x bandwidth |
| `consennet.data` | complete, tested via suite+smoke | txt/jsonl/csv/tsv/image/WAV/binary; audio is WAV-only (no codecs here) |
| `consennet.training` | complete, tested | AdamW+warmup-cosine+clip; ponder schedule + target-rounds depth loss (F3); stabilized REINFORCE (F4); resumable/wall-budgeted training (bit-identical resume) |
| `consennet.jax` | core model only | Equinox; deterministic-eval scope; outputs ≤1e-4 and round/wake counts exactly equal to torch; no JAX training |
| `consennet.hf` | complete, tested locally | config validation, bit-identical save/load, AutoModel, Trainer smoke; push_to_hub untestable (no network) |
| `consennet.fit` | complete, tested | `fit(x, y)` → trained model + honest report (accuracy, convergence, rounds/wakes/stop-p) |
| `consennet` CLI | complete, tested | `python -m consennet {info,fit,budget,panel}`; `--json` everywhere |
| `consennet.lora` | complete, tested | LoRA for stacked per-cell weights (no `nn.Linear` to wrap); zero-init attach is a bitwise no-op, merge is exact |
| `consennet.budget` | complete, tested | exact parameter-matched baselines (16 tests); replaces menu-search matching, which missed by up to 10.8% |
| `consennet.optim` | complete, tested | Muon optimizer (Newton-Schulz orthogonalized momentum, credited to Keller Jordan) adapted for stacked per-cell weights; 12 tests; measured to NOT beat AdamW at this project's scale, kept as a tested option rather than the default |
| `consennet.devpanel` | complete, tested | 12 detectors (4 new in v0.1.1: init bias, consensus saturation, during-training depth freeze, routing identity); env-gated; import-silent; weight-preserving |
| benchmarks | complete | 4 tasks × param-matched Transformers; 29 convergence runs (v0.1.1) + 28 v0.1.0 runs retained; all caveats in the reports |
| scale sweeps | complete | per-axis cost + behavior + short-train sanity |

(This table intentionally mirrors the status list in `consennet/__init__.py`.)

## Quick start

One-call training and a CLI, if you want neither the spec nor a loop:

```python
from consennet.fit import fit
rep = fit(x, y, steps=400)          # x: (N, ...), y: (N,) labels
rep["val_acc"], rep["rounds_mean"], rep["converged"]
```

```
python -m consennet info
python -m consennet fit --demo --steps 400
python -m consennet budget --target 47598
python -m consennet panel --size small
```

The full API:

```python
import torch
from consennet.spec import ConsenNetSpec
from consennet.torch.model import ConsenNet

spec = ConsenNetSpec(n_cells=4, cores_per_cell=3, d_core=32, d_signal=8,
                     d_out=10, max_rounds=8)
net = ConsenNet(spec)
x = torch.randn(16, 4, 3, 32)              # (B, cells, cores, d_core)
logits, diag = net(x)
print(diag["rounds_used"], diag["wake_events"])   # dynamics are observable
```

Multimodal (`consennet.torch.multimodal`), file loading (`consennet.data`),
training (`consennet.training.train_model`), Hugging Face
(`consennet.hf`, incl. `AutoModel` after `register_auto_classes()`), and the
JAX backend (`consennet.jax.from_torch_state_dict`) each have worked examples
in their tests — the tests are the usage documentation.

Developer diagnostics: `CONSENNET_DEV_MODE=1`, then
`from consennet.devpanel import DevPanel`.

## Reproducing the numbers

```
python -m pytest tests/                    # 148 tests
python phase0/run_experiments.py           # Phase 0 replication (E2–E6)
python scripts/speed_opt.py                # speed vs original baseline
python benchmarks/bench.py --task image --model cnA --seed 0
python benchmarks/analyze.py               # regenerate BENCHMARK_REPORT.md (v0.1.0)
python benchmarks/drive.py --wall 150      # advance the v0.1.1 job queue (resumable)
python benchmarks/analyze_v011.py          # regenerate BENCHMARK_REPORT_v0.1.1.md
python scripts/budget_audit.py             # audit parameter matching per task
python scripts/scale_sweep.py              # regenerate SCALE_REPORT.md
```

## Does dynamic depth pay? (v0.1.1 investigation)

The architecture's headline claim is that harder inputs get more computation.
Three experiments (`reports/DEPTH_INVESTIGATION.md`) find no regime in this
project where that pays: forcing the converged image models to run more rounds
gives 97.64 / 97.87 / 97.75 / 97.19 / **95.06%** at k = 2 / 3 / 4 / 6 / 8, so
the learned stopping point is near-optimal and deeper is *worse*; a
purpose-built pointer-chasing task with per-sample hop counts shows a 1-round
model matching a 6-round one with flat rounds-per-difficulty; and with the
ponder penalty at zero the model still collapses to one round. Depth
compression at convergence is a real efficiency win (2 of 8 rounds, ~2x
inference speed); adaptive depth remains undemonstrated, and making rounds
residual is the first thing v0.2 should fix.

## Architecture findings (reports/ARCHITECTURE_FINDINGS.md)

| finding | status | evidence |
|---|---|---|
| many simple units beat few complex ones | **confirmed** | 78.26% -> 84.60% at equal params, 63 sigma over 3 seeds |
| beats a Perceiver-style baseline | **confirmed** | 84.08% vs 75.26% at matched budget, 21% faster |
| cost independent of input size | **confirmed, re-sized** | flat across 64-256px; 1.5x over Perceiver (not 54x — that was vs a quadratic baseline) |
| harder inputs get more computation | **disproven** | 6 experiments; 3-SAT rounds flat at 7.98 while accuracy falls 65 points |

`consennet.fit`'s `auto_spec` shapes are now granularity-tuned as a result
(v0.1.0/v0.1.1's 4-cells-x-3-cores default sat 2.6 points below the best
measured shape). The disproven depth claim is why the architecture
description above no longer says "depth is emergent."

## Round dropout — `fit()`'s default since v0.1.2

Sampling the stopping depth during training raises image accuracy from 97.64%
to **98.09%** across three seeds at identical parameter count and identical
inference cost — above the exactly-matched Transformer's 97.31%. It was found
while investigating depth, where it failed at its intended purpose; it also
*worsens* robustness to forced extra rounds, so `fit(..., round_dropout=False)`
if you plan to call a trained model at more rounds than it trained with.
Details: `reports/DEPTH_INVESTIGATION.md`.

## Muon optimizer (`consennet.optim`, new in v0.1.2)

```python
from consennet.optim import Muon
opt = Muon.for_model(net, muon_lr=0.008, adamw_lr=3e-3)   # auto-splits params
# or: fit(x, y, optimizer="muon")
```

Adapted from Keller Jordan's Muon (Newton-Schulz orthogonalized momentum) for
ConsenNet's stacked `(n_cells, in, out)` weights — the reference iteration
already operates on the last two dimensions only, which happens to
orthogonalize each cell's slice independently with no changes needed.
Measured against AdamW+OneCycle on the granularity-tuned vision shape
(`reports/muon_comparison.json`): AdamW reaches 83.86% in 55.9s; Muon at a
tuned LR (0.008, not the published default of 0.02, which was too aggressive
here) reaches 83.68% in 69.4s — matching quality, ~24% slower per step. Muon
does not win at this project's scale, so it stays an available, tested
option rather than the default.

## Honest summary of results

Speed: batched forward 4.5-18.5x the original (fp32; 10.9-12.2x bf16 on the
larger configs); B=1 is a wash and medium-config B=1 is 0.81x (slower).

Quality (v0.1.1, trained): ConsenNet wins audio and multimodal (mean)
against parameter-matched Transformers, ties image once the baseline is
matched exactly, and loses text by 0.27 bits/char. ConsenNet also trains ~5x and infers ~6.5x faster on audio and
infers ~4x faster on text, because its cost scales with cells x rounds rather
than token count. Converged image/text models compress depth to 2 of 8 rounds
on their own; multimodal models keep all 8, where the extra rounds still buy
accuracy.

Honest limits, all detailed in the reports: text and multimodal had not
plateaued at their budget ceilings; ConsenNet's learning rates were probed and
the baselines' were not; `torch.compile` remains blocked in this environment
(five documented attempts); depth adapts across tasks but **not** per input
within a task; and separate (consensus-only) routing still fails to fuse
modalities even with 4x bandwidth or a hybrid init — the one architectural
target v0.1.1 could not hit. Failure ledgers: `reports/CLOSING_SUMMARY.md`
(v0.1.0) and `reports/CLOSING_SUMMARY_v0.1.1.md`.

## Version

0.1.2 everywhere (package, files, reports); `CHANGELOG.md` records every delta. License/status of the original
architecture remains with its author, Ömür Bera Işık.
