Metadata-Version: 2.4
Name: eden-gvhmr
Version: 1.1.1
Summary: GVHMR: World-Grounded Human Motion Recovery via Gravity-View Coordinates (packaged for use as an Eden extension)
Author: Zehong Shen and the GVHMR authors (zju3dv)
License: Copyright 2022-2023 3D Vision Group at the State Key Lab of CAD&CG,  
        Zhejiang University. All Rights Reserved. 
        
        For more information see <https://github.com/zju3dv/GVHMR> 
        If you use this software, please cite the corresponding publications   
        listed on the above website. 
        
        Permission to use, copy, modify and distribute this software and its 
        documentation for educational, research and non-profit purposes only. 
        Any modification based on this work must be open-source and prohibited 
        for commercial use. 
        You must retain, in the source form of any derivative works that you  
        distribute, all copyright, patent, trademark, and attribution notices  
        from the source form of this work. 
         
        For commercial uses of this software, please send email to xwzhou@zju.edu.cn
Project-URL: Homepage, https://zju3dv.github.io/gvhmr
Project-URL: Paper, https://arxiv.org/abs/2409.06662
Project-URL: Source, https://github.com/Kashu7100/eden_gvhmr
Project-URL: Upstream, https://github.com/zju3dv/GVHMR
Classifier: Programming Language :: Python :: 3
Classifier: License :: Other/Proprietary License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pytorch-lightning
Requires-Dist: numpy>=1.23
Requires-Dist: einops
Requires-Dist: opencv-python<5
Requires-Dist: imageio>=2.37.0
Requires-Dist: av>=13.0.0
Requires-Dist: scikit-image
Requires-Dist: termcolor
Requires-Dist: rich
Requires-Dist: joblib
Requires-Dist: colorlog
Requires-Dist: ffmpeg-python
Requires-Dist: huggingface-hub
Requires-Dist: yacs
Requires-Dist: tqdm
Requires-Dist: smplx
Requires-Dist: trimesh
Requires-Dist: cython_bbox
Requires-Dist: timm>=0.9.12
Requires-Dist: ultralytics>=8.4.0
Requires-Dist: lapx
Requires-Dist: wis3d
Requires-Dist: pycolmap
Provides-Extra: train
Requires-Dist: hydra-core>=1.3; extra == "train"
Requires-Dist: hydra-zen; extra == "train"
Requires-Dist: hydra-colorlog; extra == "train"
Requires-Dist: omegaconf; extra == "train"
Provides-Extra: dpvo
Requires-Dist: torch-scatter; extra == "dpvo"
Requires-Dist: numba; extra == "dpvo"
Requires-Dist: pypose; extra == "dpvo"
Provides-Extra: render
Provides-Extra: full
Requires-Dist: torch-scatter; extra == "full"
Requires-Dist: numba; extra == "full"
Requires-Dist: pypose; extra == "full"
Dynamic: license-file

# GVHMR: World-Grounded Human Motion Recovery via Gravity-View Coordinates
### [Project Page](https://zju3dv.github.io/gvhmr) | [Paper](https://arxiv.org/abs/2409.06662)

> World-Grounded Human Motion Recovery via Gravity-View Coordinates  
> [Zehong Shen](https://zehongs.github.io/)<sup>\*</sup>,
[Huaijin Pi](https://phj128.github.io/)<sup>\*</sup>,
[Yan Xia](https://isshikihugh.github.io/scholar),
[Zhi Cen](https://scholar.google.com/citations?user=Xyy-uFMAAAAJ),
[Sida Peng](https://pengsida.net/)<sup>†</sup>,
[Zechen Hu](https://zju3dv.github.io/gvhmr),
[Hujun Bao](http://www.cad.zju.edu.cn/home/bao/),
[Ruizhen Hu](https://csse.szu.edu.cn/staff/ruizhenhu/),
[Xiaowei Zhou](https://xzhou.me/)  
> SIGGRAPH Asia 2024

<p align="center">
    <img src=docs/example_video/project_teaser.gif alt="animated" />
</p>

## News 🔥

- [2025-03-08] By default not using DPVO. We implemented a SimpleVO, which is more efficient and compatible with GVHMR.
- [2025-03-08] We added a new option `f_mm` to specify the focal length of the fullframe camera in mm.

## Setup

Please see [installation](docs/INSTALL.md) for details.

## Quick Start

### [<img src="https://i.imgur.com/QCojoJk.png" width="30"> Google Colab demo for GVHMR](https://colab.research.google.com/drive/1N9WSchizHv2bfQqkE9Wuiegw_OT7mtGj?usp=sharing)

### [<img src="https://s2.loli.net/2024/09/15/aw3rElfQAsOkNCn.png" width="20"> HuggingFace demo for GVHMR](https://huggingface.co/spaces/LittleFrog/GVHMR)

### Demo
Demo entries are provided in `tools/demo`. Use `-s` to skip visual odometry if you know the camera is static, otherwise the camera will be estimated by DPVO.
We also provide a script `demo_folder.py` to inference a entire folder.
```shell
python tools/demo/demo.py --video=docs/example_video/tennis.mp4 -s
python tools/demo/demo_folder.py -f inputs/demo/folder_in -d outputs/demo/folder_out -s
```

The same CLI is available after `pip install` as `python -m hmr4d.demo` or the `gvhmr-demo` console script.

### Use as a library

The demo pipeline is importable, so GVHMR can be run programmatically (e.g. as an Eden extension).
The model is loaded once and reused across calls:

```python
from hmr4d.demo import GVHMR

# `checkpoint_root` points at your downloaded checkpoints (body models, GVHMR/HMR2/
# ViTPose/YOLO/DPVO weights). It defaults to ./inputs/checkpoints; set it (or the
# $GVHMR_CHECKPOINT_ROOT env var) to run from any working directory.
model = GVHMR(checkpoint_root="/path/to/inputs/checkpoints")
out = model.recover("docs/example_video/tennis.mp4", static_cam=True)  # render=True for overlay videos
# out -> {"smpl_params_global", "smpl_params_incam", "K_fullimg", "output_dir"}
```

GVHMR is CUDA-only (select a GPU with `CUDA_VISIBLE_DEVICES`). It composes its own Hydra
config, so it must run in a process without an already-initialized Hydra — when embedding it
in another Hydra-using application (such as Eden), run it out-of-process.

### Reproduce
1. **Test**:
To reproduce the 3DPW, RICH, and EMDB results in a single run, use the following command:
    ```shell
    python tools/train.py global/task=gvhmr/test_3dpw_emdb_rich exp=gvhmr/mixed/mixed ckpt_path=inputs/checkpoints/gvhmr/gvhmr_siga24_release.ckpt
    ```
    To test individual datasets, change `global/task` to `gvhmr/test_3dpw`, `gvhmr/test_rich`, or `gvhmr/test_emdb`.

2. **Train**:
To train the model, use the following command:
    ```shell
    # The gvhmr_siga24_release.ckpt is trained with 2x4090 for 420 epochs, note that different GPU settings may lead to different results.
    python tools/train.py exp=gvhmr/mixed/mixed
    ```
    During training, note that we do not employ post-processing as in the test script, so the global metrics results will differ (but should still be good for comparison with baseline methods).

# Citation

If you find this code useful for your research, please use the following BibTeX entry.

```
@inproceedings{shen2024gvhmr,
  title={World-Grounded Human Motion Recovery via Gravity-View Coordinates},
  author={Shen, Zehong and Pi, Huaijin and Xia, Yan and Cen, Zhi and Peng, Sida and Hu, Zechen and Bao, Hujun and Hu, Ruizhen and Zhou, Xiaowei},
  booktitle={SIGGRAPH Asia Conference Proceedings},
  year={2024}
}
```

# Acknowledgement

We thank the authors of
[WHAM](https://github.com/yohanshin/WHAM),
[4D-Humans](https://github.com/shubham-goel/4D-Humans),
and [ViTPose-Pytorch](https://github.com/gpastal24/ViTPose-Pytorch) for their great works, without which our project/code would not be possible.
