Metadata-Version: 2.4
Name: gesto
Version: 0.2.5
Summary: Train and run gesture recognition models from Gesto Labeller datasets
Author: Sundar Balamurugan
License: MIT
Project-URL: Homepage, https://gestostudio.com
Project-URL: Documentation, https://gestostudio.com/docs.html
Project-URL: Repository, https://github.com/TheMadrasTechie/gesto
Project-URL: Issues, https://github.com/TheMadrasTechie/gesto/issues
Keywords: gesture-recognition,mediapipe,pose-estimation,computer-vision,lstm,sign-language
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Image Recognition
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy<2,>=1.23
Requires-Dist: protobuf<5,>=3.20
Requires-Dist: tensorflow<2.18,>=2.15
Requires-Dist: opencv-python<4.12,>=4.8
Requires-Dist: mediapipe<0.10.30,>=0.10
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

<p align="center">
  <img src="https://raw.githubusercontent.com/TheMadrasTechie/gesto/master/assets/gesto.svg" alt="Gesto logo" width="120" height="120"/>
</p>

<h1 align="center">gesto</h1>

<p align="center">
  Train and run gesture recognition models from Gesto Labeller datasets.
</p>

<p align="center">
  <img src="https://img.shields.io/pypi/v/gesto" alt="PyPI version"/>
  <img src="https://img.shields.io/pypi/pyversions/gesto" alt="Python versions"/>
  <img src="https://img.shields.io/github/license/TheMadrasTechie/gesto" alt="License"/>
  <img src="https://img.shields.io/pypi/dm/gesto" alt="Downloads"/>
</p>

<p align="center">
  <a href="https://gestostudio.com">Website</a> ·
  <a href="https://gestostudio.com/docs.html">Docs</a> ·
  <a href="https://github.com/TheMadrasTechie/gesto">GitHub</a>
</p>

<p align="center">
  <code>pip install gesto</code>
</p>

Point it at a project folder, pick a mode and a region, and it handles the rest —
loading, training, versioned model storage, and live detection that matches how
the data was captured.

## Two modes

| mode | the gesture is… | model | example |
|---|---|---|---|
| `static` | a held shape or posture | Dense network | thumbs up, alphabet letters, a stance |
| `sequence` | a motion over time | stacked LSTM | waving, clapping, jogging |

Static needs far less data (every captured frame is a training sample) and
predicts instantly with no warm-up. Reach for `sequence` only when two gestures
share the same shape and differ by movement.

## Five regions

| region | dim | tracks |
|---|---|---|
| `hands_one` | 63 | one hand, 21 joints |
| `hands_two` | 126 | both hands |
| `pose` | 132 | full body, 33 points |
| `legs` | 32 | lower body, 8 points |
| `full` | 258 | body + both hands |

The region must match how the project was captured — `gesto` checks the feature
dimension and tells you if it doesn't.

## Command line

Two equivalent styles. Use whichever you like.

**General** — mode and region as arguments:

```bash
gesto train static hands_one ./gesto_projects/signs
gesto detect sequence pose --source clip.mp4
gesto image hands_one photo.jpg
```

**Per-combination** — one command per mode+region (there's one for each):

```bash
gesto train-static-legs ./gesto_projects/stances --epochs 250
gesto detect-sequence-pose --source clip.mp4
gesto image-static-hands-one photo.jpg
```

### Detecting on camera vs video

`--source` takes a webcam index or a file path:

```bash
gesto detect static hands_one                       # default webcam (index 0)
gesto detect static hands_one --source 1            # second camera
gesto detect sequence pose --source walk.mp4        # a video file
gesto detect sequence pose --source C:\clips\run.avi
```

### Classifying a single image (static models)

```bash
gesto image hands_one photo.jpg                     # opens a window with the result
gesto image hands_one photo.jpg --no-show           # just print the prediction
gesto image-static-pose posture.png --version 2     # a specific model version
```

### Drawing landmarks

Landmarks are drawn on the frame by default. Turn them off with `--no-draw`:

```bash
gesto detect static hands_one                        # skeleton drawn (default)
gesto detect static hands_one --no-draw              # clean video, no skeleton
gesto image hands_one photo.jpg --no-draw
```

### Training options

Epochs and other hyperparameters are adjustable on any train command:

```bash
gesto train sequence pose ./proj --epochs 400 --batch-size 32 --seq-len 30
gesto train-static-hands-one ./proj --epochs 150 --large   # force full model
```


## Python

```python
import gesto

run = gesto.train("./gesto_projects/signs", region="hands_one", mode="static")
gesto.detect("static", "hands_one")                       # camera
gesto.detect("sequence", "pose", source="clip.mp4")       # video

# classify a single image with a static model
from gesto.detect import predict_image
label, confidence, probs = predict_image("static", "hands_one", "photo.jpg",
                                         show=False, draw_landmarks=False)
```

Or drive a model yourself:

```python
from gesto.detect import Predictor

predictor = Predictor.load("static", "hands_one")
vector = predictor.features(holistic_result)   # extract + normalize
probs = predictor.predict(vector)
```

## Where models go

Everything lands under one `artifacts/` folder, split by mode then region.
Training never overwrites an earlier run — it versions:

```
artifacts/
    static/
        hands_one/          model.keras, labels.json
        hands_one_2/        the next run
        pose/
    sequence/
        pose/
        pose_2/
```

`gesto detect static pose` picks the newest version; `--version 1` picks a
specific one.

## Matching your capture

Predictions are only correct when detection feeds the model the same kind of
vector it trained on. `gesto` mirrors Gesto Labeller exactly:

- **the same engine** — MediaPipe Holistic, same confidence settings
- **the same landmark order** — including one-hand mode preferring the right
  hand and falling back to the left
- **the same normalization** — translation/scale-invariant, verified identical
- **the same mirroring** — webcam frames are flipped, video files are not

If you captured with Gesto's **Normalise** unchecked, pass `--raw` when training
so detection knows to skip it.

## Getting good results

- **Balance your classes.** Similar sample counts per class; `gesto` applies
  class weights but balanced data is better.
- **Enough samples.** ~20–30 static frames per class, or ~15–30 sequences.
  Small datasets automatically get a lighter model, since an oversized network
  on little data overfits and collapses to predicting one class.
- **Consistent clip length** for sequence mode — set "Max frames" in Gesto
  Labeller so every capture is the same length.

Run `gesto inspect <project>` to check all of this before training.

## Installation

```bash
pip install gesto
```

`gesto` uses MediaPipe's legacy `solutions` API, which was removed in MediaPipe
0.10.31. It also needs versions of TensorFlow, NumPy and protobuf that agree
with that MediaPipe — newer TensorFlow (2.21+) and OpenCV (5.0) pull protobuf
and NumPy in an incompatible direction. The package therefore pins a coherent,
tested set:

| package | pinned range | tested with |
|---|---|---|
| mediapipe | `>=0.10,<0.10.30` | 0.10.21 |
| tensorflow | `>=2.15,<2.18` | 2.17.1 |
| numpy | `>=1.23,<2` | 1.26.4 |
| protobuf | `>=3.20,<5` | 4.25.9 |
| opencv-python | `>=4.8,<4.12` | 4.11.0 |

Install into a **fresh virtual environment** so these don't clash with other
projects:

```bash
python -m venv gesto_env
# Windows:  gesto_env\Scripts\activate
# macOS/Linux:  source gesto_env/bin/activate
pip install gesto
```

If you already hit dependency conflicts (e.g. you had TensorFlow 2.21 or
OpenCV 5.0 installed), the cleanest fix is a fresh venv as above. To repair an
existing environment, pin the set explicitly:

```bash
pip install "mediapipe==0.10.21" "tensorflow==2.17.1" "numpy==1.26.4" "protobuf==4.25.9" "opencv-python==4.11.0.86"
```

## Roadmap

The legacy MediaPipe `solutions` API won't be maintained forever. A future
release will move to MediaPipe's newer **Tasks** API (`HandLandmarker`,
`PoseLandmarker`), which lifts the version ceiling. That API produces slightly
different hand-landmark geometry, so models would need retraining — hence it's a
deliberate, separate step rather than a drop-in change.

## License

MIT
