Metadata-Version: 2.3
Name: cosmo-torrent
Version: 0.1.0
Summary: Data downloader library for Cosmology related data
Project-URL: Repository, https://cosmo-docs.phys.ethz.ch/cosmo-torrent
Author-email: Uwe Schmitt <uwe.schmitt@id.ethz.ch>
License: MIT License
License-File: AUTHORS.rst
License-File: LICENSE
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Dist: click
Requires-Dist: requests
Requires-Dist: tqdm
Provides-Extra: dev
Requires-Dist: isort; extra == 'dev'
Requires-Dist: pre-commit; extra == 'dev'
Requires-Dist: pytest; extra == 'dev'
Requires-Dist: pytest-cov; extra == 'dev'
Requires-Dist: pytest-regtest; extra == 'dev'
Requires-Dist: ruff; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: docs
Requires-Dist: myst-parser; extra == 'docs'
Requires-Dist: sphinx-pynpoint-theme; extra == 'docs'
Requires-Dist: sphinx<7; extra == 'docs'
Description-Content-Type: text/markdown

# cosmo-torrent

This project offers a library to access data from central storage location
and a command line tool to upload data.

This can be used to avoid shipping large data sets within Python packages.

## Data upload

After installing `cosmo-torrent` the command line command

```
$ cosmo-torrent-upload --help
Usage: cosmo-torrent-upload [OPTIONS] FOLDER

Options:
  --identifier TEXT  data set identifier for later download  [required]
  --user TEXT        user name for scp to server
  --force            overwrite existing dataset
  --help             Show this message and exit.
```

can be used to upload data to a central storage location (details in the
[Internals](#Internals) section below).

For example:

```
$ cosmo-torrent-upload --identifier test_data.2024.08 --user schmittu .
```
uploads the content of the current working folder using the identifier
`test_data.2024.08`.

## Data accesss

The data can be accessed using the identifier used for the upload.
On first access, the data is downloaded to a local cache folder.
For future access, the dataset can be accessed directly.


```python
from cosmo_torrent import data_set

test_data_folder = data_set('test_data.2024.07')
```

Here `test_data_folder` will now be a local folder with the specified data set.



## Internals

The data is stored at
`/net/ipa-gate.phys.ethz.ch/export/ipa/public_html/cosmo-torrent` at the DPHYS NAS
which can be accessed via https://share.phys.ethz.ch/~ipa/cosmo-torrent
