Metadata-Version: 2.1
Name: tspace
Version: 0.0.1
Summary: io interface
Home-page: https://github.com/Binjian/tspace
Author: binjian xin
Author-email: binjian.xin@hotmail.com
License: Apache Software License 2.0
Keywords: nbdev jupyter notebook python
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.11
Classifier: License :: OSI Approved :: Apache Software License
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastcore
Requires-Dist: pandas
Requires-Dist: numpy
Requires-Dist: pydantic
Requires-Dist: pyarrow
Requires-Dist: ordered-set
Requires-Dist: black[jupyter]
Requires-Dist: ruff
Requires-Dist: mypy
Requires-Dist: pylint
Requires-Dist: typing_inspect
Requires-Dist: typeguard
Requires-Dist: dacite
Requires-Dist: poetry
Requires-Dist: tensorflow
Requires-Dist: tensorflow-estimator
Requires-Dist: tensorflow-probability
Requires-Dist: tensorboard
Requires-Dist: flatbuffers
Requires-Dist: keras
Requires-Dist: python-json-logger
Requires-Dist: plotly
Requires-Dist: Pillow
Requires-Dist: pandas
Requires-Dist: opt-einsum
Requires-Dist: jupyterlab
Requires-Dist: notebook
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: seaborn
Requires-Dist: scikit-learn
Requires-Dist: matplotlib
Requires-Dist: pytest
Requires-Dist: reportlab
Requires-Dist: pymongo
Requires-Dist: fastavro
Requires-Dist: gitpython
Requires-Dist: jupytext
Requires-Dist: tqdm
Requires-Dist: pandas-stubs
Requires-Dist: nbstripout
Requires-Dist: nbdev
Requires-Dist: matplotlib-stubs
Requires-Dist: pyarrow
Requires-Dist: dask
Requires-Dist: typeguard
Requires-Dist: poetry2conda
Requires-Dist: cutelog
Requires-Dist: pdoc3
Requires-Dist: itikz
Provides-Extra: dev

# tspace


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

# Overview

tspace is an data pipleline framework for deep reinforcement learning
with IO interface, processing and configuration

- Working in training and inferrence mode
  - logging and monitoring with cutelog or TUI interface
  - cascaded threading pool for well-structured Scheduling of
    [ETL](https://en.wikipedia.org/wiki/Extract,_transform,_load) and ML
    pipelines
  - Customized Exception handling
  - Graceful shutdown
  - online and offline training
  - local and distributed training
- Support for multiple models
  - reinforcement learning models with DDPG
  - time sequence models with LSTM and Transformer
- Data pipeline compatible to both ETL and ML dataflow
  - Support for multiple data sources (local CAN or remote cloud object
    storage)
  - Support both NoSQL database and local or cloud data storage through
    Dask with Parquet and Avro interface
  - Full Pandas DataFrame support with raw json codecs
  - Configuration system for vehicles, drivers, data sites, neural
    network hyperparameters, database, HMI types, etc
  - Timezone aware time sequence data processing
  - Data object meta-info processing and storage linked to configuration
    system
  - Stateful time sequence processing with sequential model
  - Type hint for data processing and configuration
  - Pydantic integration

<img src="res/tspace_overview.svg" alt="Overview of tspace architecture" width="80%">

The diagram shows the basic architure of tspace. The main components
are:

- **Avatar**: orchestrates the whole ETL and ML workflow.
- **Data Object**: The data object is a container for the data. It
  contains the raw data, the metainfo and the configuration. The data
  object is passed through the data pipeline and can be modified by the
  processing components.
- **Data Pipeline**: The data pipeline is a sequence of processing
  components. Each processing component takes a data object as input and
  returns a data object as output. The data pipeline is responsible for
  transforming the data object into a format that can be used by the
  machine learning model.
- **Machine Learning Model**: The machine learning model is responsible
  for training and inferencing. It takes the data object as input and
  returns a prediction as output. The machine learning model can be a
  reinforcement learning model, a time sequence model or any other type
  of model.
- **Configuration System**: The configuration system is responsible for
  storing and managing the configuration of the data pipeline, the
  machine learning model and other components. The configuration system
  is used to configure the data pipeline, the machine learning model and
  other components.
- **Logging and Monitoring**: The logging and monitoring component is
  responsible for logging and monitoring the data pipeline, the machine
  learning model and other components. It provides real-time feedback on
  the performance of the data pipeline and the machine learning model.
- **Exception Handling**: The exception handling component is
  responsible for handling exceptions that occur during the execution of
  the data pipeline, the machine learning model and other components. It
  provides a mechanism for gracefully handling errors and recovering
  from failures.

## TODO

1.  Add time sequence embedding database support with LanceDB for
    TimeGPT
2.  Batch mode for large scale inference and training with Unit of Work
    pattern

# How to use

## Install

``` sh
pip install tspace
```

Fill me in please! Don’t forget code examples:

``` python
1+1
```

    2

``` python
# say_hi("world")
foo()
```

    'foo'
