Metadata-Version: 2.4
Name: rb-deeplearning-lib
Version: 1.0.5
Summary: This is a machine learning--more specifically deep learning--library from my independent study on deep learning. This library is both a result of my learning and a tool for AI development.
License-Expression: MIT
Project-URL: Homepage, https://github.com/rylan-berry/DeepLearningIndependentStudy/tree/main/deeplearning_package
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=2.0
Dynamic: license-file

# Deeplearning Package

## Overview

This package is designed to be similar to the PyTorch system of a building block system, providing functions that can be mixed, matched, and customized for any given model. It implements core deep learning concepts including automatic differentiation, sequential modeling, various layer types (CNN, RNN, Attention), optimization algorithms, and utilities for model saving and loading.

## Modules

This project contains several key modules:

*   `autogradient.py`
*   `sequence.py`
*   `optimizer.py`
*   `neural_net.py`
*   `cnn_layers.py`
*   `rnn_layers.py`
*   `attention.py`
*   `save_load_funcs.py`

All of which are automatically part of the initial import of the package.

## Detailed Module Descriptions

### `autogradient.py`

This module forms the core of the automatic differentiation system. It introduces the `Values` class, which tracks computational history to compute gradients.

*   **`Values` Class**: Encapsulates numerical values (`vals`) and their corresponding gradients (`grad`). It supports a wide array of operations and tensor manipulations.
*   **Broadcasting Support**: Includes a sophisticated `_broadcast_grad` static method that automatically handles gradient shape adjustments during backpropagation when operations involve broadcasted NumPy arrays (e.g., adding a bias vector to a matrix or operations with varying input dimensions).
*   **Operations**: Supports a comprehensive set of operations including arithmetic (`+`, `-`, `*`, `/`, `@`, `**`), activation functions (`relu`, `tanh`, `softmax`, `log`, `exp`), and crucial tensor manipulations (`reshape`, `transpose`, `pad`, `sum`, `mean`, `max`, `abs`, and flexible slicing/indexing via `__getitem__` and `__setitem__`). It also provides direct access to the NumPy `T` (transpose) attribute.

### `rnn_layers.py`

This module provides implementations for Recurrent Neural Networks, suitable for sequential data processing.

*   **`FullRNNLayer`**: Implements a standard RNN layer with configurable input, hidden, and output dimensions. It supports different activation functions (`tanh` by default) and handles input sequence transposition for flexible batching. The `_rnn_cell_forward` method details the computation for a single time step, integrating input, previous hidden state, and biases to produce the current hidden state and output.

### `cnn_layers.py`

Implements fundamental building blocks for Convolutional Neural Networks.

*   **`Reshape`**: Enables built in reshaping, uses `np.ndarray.reshape(...)`, and takes the `size` as the input. Backpropagation is enabled for it.
*   **`Convo2D`**: Supports 2D convolution with configurable `kernel_matrix`, `padding` ('valid' or 'same'), and `stride`. It automatically calculates output dimensions and applies padding. Gradients are computed for the kernel during backpropagation.
*   **Pooling**: Includes `MaxPooling` and `AvgPooling` classes, which perform spatial down-sampling operations with specified `pool_size` and `stride`.

### `neural_net.py`

Defines high-level components for neural network construction, training, and regularization.

*   **`Layer`**: Represents a single fully connected layer with weights, biases, and a customizable activation function (e.g., `relu`, `softmax`, `tanh`) dynamically accessed via `getattr`.
*   **`Dense`**: A multi-layered dense block, composed of multiple `Layer` instances, allowing for easy creation of deep fully connected networks.
*   **`Dropout`**: A regularization technique to prevent overfitting by randomly setting a fraction of input units to zero during training.
*   **`Model`**: The central class for defining, training, and evaluating neural networks. It wraps a `Sequence` of blocks, manages an `optimizer`, and handles the training loop, batching, and integration with various `loss_fn` (e.g., `cross_entropy_loss`, `mse_loss`) and `pen_fn` (e.g., `l1_reg`, `l2_reg`).
*   **Loss Functions**: `cross_entropy_loss` and `mse_loss` are provided for common supervised learning tasks.
*   **Penalty Functions**: `l1_reg` (L1 regularization) and `l2_reg` (L2 regularization) are available to apply penalties to model weights during training, helping to prevent overfitting.

### `optimizer.py`

Implements various parameter update algorithms to minimize the loss function during training.

*   **`Optimizer` (Base Class)**: Defines the common interface for all optimizers.
*   **`Optim_SGD`**: Stochastic Gradient Descent with an optional learning rate scheduler (`finitters`, `fin_l_rate`).
*   **`Optim_SGD_Momentum`**: SGD with momentum to accelerate convergence and dampen oscillations.
*   **`Optim_AdaGrad`**: Adaptive Gradient algorithm, which scales learning rates for each parameter individually.
*   **`Optim_RMSPropclass`**: RMSProp optimizer, an adaptive learning rate method that divides the learning rate by an exponentially decaying average of squared gradients.
*   **`Optim_Adam`**: Adaptive Moment Estimation, combining ideas from Momentum and RMSProp for efficient optimization.

### `sequence.py`

Provides the `Sequence` container to stack layers and manage parameter retrieval and setting across the model. It allows for a straightforward, sequential construction of neural networks.

### `attention.py`

This module implements mechanisms for attention, crucial for modern sequence models like Transformers.

*   **`Embedding`**: Converts discrete tokens (elements) into dense vector representations. It manages a vocabulary (`encoder`, `decoder`) and provides functionality to `encode` elements to indices, `decode` indices back to elements, and generate embeddings for input sequences. It supports both raw elements and pre-encoded indices as input.
*   **`AttentionHead`**: The core component of self-attention. It projects input features into Query (Q), Key (K), and Value (V) representations, calculates attention scores (`S`) via dot product, normalizes them using `softmax` to get `A` (attention weights), and computes the weighted sum of Value vectors (`Y`).
*   **`AttentionMultiHead`**: Combines multiple `AttentionHead` instances in parallel. It concatenates the outputs of individual heads and projects them through a final linear layer (`w_0`), allowing the model to jointly attend to information from different representation subspaces at different positions.

### `save_load_funcs.py`

This module provides utility functions for saving and loading model parameters, enabling persistence of trained models.

*   **`getModelSave`**: Extracts the numerical values of all trainable parameters (weights and biases) from a given model and converts them into a list of standard Python lists (or NumPy arrays), suitable for serialization (e.g., to JSON, Pickle, or other formats).
*   **`modelLoadParams`**: Takes a list of parameter matrices (either NumPy arrays or `Values` objects) and loads them back into a `Model` instance, setting the model's weights and biases to the provided values. This allows for reloading previously saved model states.

## Dependencies

The package relies heavily on `numpy` for numerical operations and gradient tracking.

## Version 1.0.0 Notice

The addition of this updated description marks the end of the primary development of this package (and the end of my independent study course on this topic). Any 1.0.x will be simple bug fixes to this release. If I do continue to update this in the future they'll be under 1.x versions. 
