Metadata-Version: 2.1
Name: shopcloud-datalake
Version: 1.3.0
Summary: CLI tool for datalake operation
Home-page: https://github.com/Talk-Point/shopcloud-datalake-cli
Author: Konstantin Stoldt
Author-email: konstantin.stoldt@talk-point.de
License: MIT
Keywords: CLI
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pyyaml
Requires-Dist: requests
Requires-Dist: pytest
Requires-Dist: shopcloud-secrethub >=2.12.0
Requires-Dist: pre-commit
Requires-Dist: google-cloud-storage
Requires-Dist: google-cloud-bigquery
Requires-Dist: pandas
Requires-Dist: tqdm
Requires-Dist: pyarrow
Requires-Dist: fastparquet
Requires-Dist: prompt-toolkit

# shopcloud-datalake-cli
Datalake ETL tool

```sh
mkdir config-dir
$ datalake --project="evident-castle-411608" --base-dir="examples" config create
$ datalake --project="evident-castle-411608" --base-dir="examples" config sync
$ datalake --project="evident-castle-411608" --base-dir="examples" run --partition-date=2024-01-19
$ datalake --project="evident-castle-411608" --base-dir="examples" run <table> --partition-date=2024-01-19
```

## Development

```sh
# run unit tests
$ python3 -m unittest
# run unit tests with coverage
$ python3 -m coverage run --source=tests,shopcloud_datalake -m unittest discover && python3 -m coverage html -d coverage_report
$ python3 -m coverage run --source=tests,shopcloud_datalake -m unittest discover && python3 -m coverage xml
```
