Metadata-Version: 2.4
Name: tgedr-datasets
Version: 0.1.11
Summary: datasets to be consumed elsewhere
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: pandas~=2.3.0
Requires-Dist: tgedr-dataops==1.0.5
Requires-Dist: pyarrow~=23.0.0
Requires-Dist: lxml>=6.0.2
Requires-Dist: html5lib>=1.1
Requires-Dist: bs4>=0.0.2
Requires-Dist: tgedr-pycommons>=1.0.1
Requires-Dist: yfinance==0.2.66
Requires-Dist: finnhub-python>=2.4.26
Requires-Dist: matplotlib>=3.10.8
Requires-Dist: datasets>=4.5.0

# datasets

[![cicd](https://github.com/jtviegas/datasets/actions/workflows/cicd.yml/badge.svg)](https://github.com/jtviegas/datasets/actions/workflows/cicd.yml)
![Coverage](./coverage.svg)
[![PyPI](https://img.shields.io/pypi/v/tgedr-dataset)](https://pypi.org/project/tgedr-dataset/)

## datalake pipelines


[![tickers](https://github.com/jtviegas/datasets/actions/workflows/tickers.yml/badge.svg)](https://github.com/jtviegas/datasets/actions/workflows/tickers.yml)
[![prices](https://github.com/jtviegas/datasets/actions/workflows/prices.yml/badge.svg)](https://github.com/jtviegas/datasets/actions/workflows/prices.yml) 
[![articles](https://github.com/jtviegas/datasets/actions/workflows/articles.yml/badge.svg)](https://github.com/jtviegas/datasets/actions/workflows/articles.yml)

### articles

![articles distribution per ticker](assets/article_distribution.png)

![number of articles found per run](assets/articles_per_processing_time.png)



## development
- main requirements:
  - _uv_  
  - _bash_
- Clone the repository like this:

  ``` bash
  git clone git@github.com:jtviegas/datasets
  ```
- cd into the folder: `cd datasets`
- install requirements: `./helper.sh reqs`
