Metadata-Version: 2.4
Name: pylafs
Version: 0.1.0
Summary: PyLAFS — Learning Attention-based Feature Selection: a PyTorch-powered, scikit-learn compatible feature selector using trainable attention.
Author: Hincal Topcuoglu
License-Expression: MIT
Project-URL: Homepage, https://github.com/hincaltopcuoglu/pylafs
Project-URL: Documentation, https://github.com/hincaltopcuoglu/pylafs#readme
Project-URL: Repository, https://github.com/hincaltopcuoglu/pylafs
Keywords: feature-selection,attention,deep-learning,machine-learning,pytorch,scikit-learn
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.21
Requires-Dist: pandas>=1.3
Requires-Dist: scikit-learn>=1.0
Requires-Dist: torch>=1.10
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Provides-Extra: benchmark
Requires-Dist: xgboost; extra == "benchmark"
Requires-Dist: pmlb; extra == "benchmark"
Dynamic: license-file

# PyLAFS — Learning Attention-based Feature Selection

A PyTorch-powered, **scikit-learn compatible** feature selection method that uses a trainable attention mechanism to rank and select the most informative features for classification tasks.

## Installation

```bash
pip install pylafs
```

Or in development/editable mode:

```bash
pip install -e .
```

## Quick Start

```python
from pylafs import LAFSSelector
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import pandas as pd

# Generate sample data
X, y = make_classification(n_samples=1000, n_features=50,
                           n_informative=10, random_state=42)
X = pd.DataFrame(X, columns=[f"f_{i}" for i in range(50)])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Scale features
scaler = StandardScaler()
X_train = pd.DataFrame(scaler.fit_transform(X_train), columns=X.columns)
X_test  = pd.DataFrame(scaler.transform(X_test), columns=X.columns)

# Select top-10 features with LAFS
selector = LAFSSelector(k=10, epochs=50, lr=0.001, lambda_reg=0.01)
selector.fit(X_train, y_train)

X_train_selected = selector.transform(X_train)
X_test_selected  = selector.transform(X_test)

print("Selected features:", selector.selected_features_)
print("Feature importances:", selector.feature_importances_)
```

## API Reference

### `LAFSSelector`

| Parameter | Type | Default | Description |
|---|---|---|---|
| `k` | int | 10 | Number of features to select |
| `lr` | float | 0.001 | Adam learning rate |
| `lambda_reg` | float | 0.01 | Entropy regularisation coefficient |
| `epochs` | int | 50 | Training epochs |
| `batch_size` | int | 256 | Mini-batch size |
| `attn_hidden_size` | int | 128 | Attention network hidden layer width |
| `attn_dropout_rate` | float | 0.5 | Dropout rate in attention network |
| `cls_hidden_size` | int | 128 | Classifier hidden layer width |
| `weight_decay` | float | 1e-4 | L2 regularisation for Adam |
| `device` | str/None | None | `"cuda"`, `"cpu"`, or auto-detect |
| `random_state` | int/None | None | Seed for reproducibility |
| `verbose` | bool | False | Print training progress |

### Methods

| Method | Description |
|---|---|
| `fit(X, y)` | Train the LAFS model and compute feature importances |
| `transform(X)` | Reduce X to the selected features |
| `fit_transform(X, y)` | Fit and transform in one step |
| `get_support(indices=False)` | Boolean mask or indices of selected features |
| `get_feature_names_out()` | Names of the selected features |

### Attributes (available after `fit`)

| Attribute | Description |
|---|---|
| `feature_importances_` | Mean attention weight per feature (numpy array) |
| `ranking_` | Feature indices sorted by importance (descending) |
| `selected_features_` | List of selected feature names |
| `selected_indices_` | Array of selected feature indices |
| `model_` | The trained PyTorch `LAFSModel` |

## How It Works

LAFS jointly trains two neural networks:

1. **AttnNet** — produces per-feature softmax attention weights
2. **ClassNet** — a feedforward classifier that operates on attention-weighted features

The loss function combines cross-entropy classification loss with an entropy regularisation term on the attention weights. This encourages the attention to focus on the most discriminative features.

After training, global feature importance is computed as the mean attention weight across all training samples. The top-*k* features by importance are selected.

## License

MIT
