Metadata-Version: 2.4
Name: YURIformer
Version: 0.1.0
Summary: A vibe-coded, hierarchical dendritic computing framework.
Author: moelanoby
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.0.0
Requires-Dist: triton>=2.1.0
Requires-Dist: numpy

# YURIformer :3

heya everyone :] welcome to YURIformer! this entire architecture is aggressively vibe coded, meaning I used AI to help me mostly with the code to prototype faster [and i cant code] but all the things like triton kernels and DEQ solver algorithims are verified by me :3

tho it was designed to be a modular library for efficiency and innovative learning rules, neural nets, and efficient numerical operations :] 

---

### [Written by a human] (Q&A Time! :3)
**What is the purpose of the project?**

The purpose of this project was to simplify the implementation of innoative learning rules and architectures :]
currently I have implemented DEQ kernels for learning and posit 16 kernels for our DEQs and numeric operations :3 

**What is under development?**

- I will explore neuromorphic computing next like adding OSTL , OSTTP, OTTT, and much more ya can use :3
- I will also explore niche parts of ML beyond just these 2
- While i did mention i will add neural networks they are sadly under progress :[

**How/What to contribute?**
 - while this project IS vibecoded [tho i am learnibng to write code by hand!] I encourage people to contribute using their own sweats and tears [aka written by real human :) ]
 - I also encourage opening issues for bugs or errors that i could try to fix well 
 - I also heavily encourage you to make PRs since we need more human written code more human creativity :]

---

### [Generated by an AI]

**Installation**

Since the project is structured properly with a `pyproject.toml`, you can easily install it locally. Just clone the repo and install it in editable mode :3

```bash
git clone https://github.com/moelanoby/YURIformer.git
cd YURIformer
pip install -e .
```

This will automatically install dependencies like `torch`, `triton`, and `numpy`, making all the `architecture_kernels`, `numeric_kernels`, and `learning_rules` importable from anywhere in your environment :D

**How to Use DEQ Kernels (Drop-in Replacement for Infinite Depth)**

We have a fully modular Deep Equilibrium (DEQ) library that replaces Backpropagation Through Time (BPTT) for implicit *depth* (weight-tied layers), rather than sequence time. It effectively gives you an infinite-depth network with O(1) memory and super fast training! :]

```python
import torch
import torch.nn as nn
from learning_rules.DEQ_kernels import DEQModule, HybridConfig, SolverFactory

# 1. Define your recurrent cell (the exact same one you use for BPTT!)
class MyCell(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.fc = nn.Linear(dim, dim)
        self.norm = nn.LayerNorm(dim)
        
    def forward(self, z, x):
        return torch.tanh(self.norm(self.fc(z) + x))

# 2. Pick a solver config (e.g., Hybrid, Anderson, Broyden, or PJWR) :p
cfg = HybridConfig(max_iter=40, tol=1e-4)

# 3. Wrap your cell in the DEQModule :3
cell = MyCell(dim=64)
solver = SolverFactory.create(cfg, cell)
deq_layer = DEQModule(cell, solver=solver, backward_mode='phantom')

# 4. Forward pass finds the fixed point implicitly! :D
x = torch.randn(32, 64)
z_star = deq_layer(x)
```

**Wanna see it in action? Check out the drop-in replacement example!** :D

We wrote a super detailed script that shows you *exactly* how to transition your codebase from BPTT to DEQ! Go open up `examples/deq_dropin_replace_bptt.py` and give it a read :3 It has five different patterns showing you:
- The minimal swap (changing just 4 lines of code!)
- How to pick your solver dynamically using config dataclasses
- How to switch between `phantom` and `neumann-1` backward modes
- How to add Jacobian Regularization for extra stability! :p

Seriously, it's fully documented and runnable. Try running `python examples/deq_dropin_replace_bptt.py` to see the side-by-side performance benchmarks happen live on your machine :]

**Benchmarks**

We benchmarked our DEQ solvers against standard BPTT on a synthetic task (30 training steps). Using our custom solvers with phantom gradients, we achieve great accuracy with O(1) memory scaling while avoiding the deep unrolling overhead :p

| Training Method | Memory Scaling | Speed (30 steps) | Accuracy | Where they break (usually) |
| :--- | :--- | :--- | :--- | :------------ |
| **Standard BPTT** (unroll=5) | O(N) | 0.12s | 100% | its O(L) memory cost |
| **Standard BPTT** (unroll=20) | O(N) | 0.11s | 92% | its O(L) memory cost |
| **DEQ (PJWR)** | O(1) | **0.03s** | 55% | it is fast! but falls short in accuracy if not intialized correctly |
| **DEQ (Broyden)** | O(1) | 0.19s | 55% | without anderson warm up it also falls short in accuracy |
| **DEQ (Hybrid)** | O(1) | 0.27s | **100%** | yes it is fast but might fall short in other benchmarks that this doesnt show but not as much as the previous 2 |
| **DEQ (Anderson)** | O(1) | 0.29s | **100%** | THE INDUSTRY STANDARD but CAN be a bit slower than the hybrid in long runs |

*Note: The solvers find the fixed point z* implicitly, bypassing the need to store activations for the entire depth history! :]*

