Metadata-Version: 2.1
Name: text2mapviewer
Version: 0.2.0
Summary: A python package to map your own csv files data using Atlas from NOMIC
Author-email: Papa Séga WADE <pasega.wade@gmail.com>
Project-URL: Homepage, https://github.com/papasega/text2mapviewer
Keywords: Embedding, Visualization, Map, Text, CSV,  Search keywords, Dynamic
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: aiohttp (==3.8.4)
Requires-Dist: aiosignal (==1.3.1)
Requires-Dist: async-timeout (==4.0.2)
Requires-Dist: attrs (==22.2.0)
Requires-Dist: backoff (==2.2.1)
Requires-Dist: certifi (==2022.12.7)
Requires-Dist: charset-normalizer (==3.1.0)
Requires-Dist: click (==8.1.3)
Requires-Dist: cmake (==3.26.3)
Requires-Dist: cohere (==4.1.4)
Requires-Dist: filelock (==3.11.0)
Requires-Dist: frozenlist (==1.3.3)
Requires-Dist: huggingface-hub (==0.13.4)
Requires-Dist: idna (==3.4)
Requires-Dist: jinja2 (==3.1.2)
Requires-Dist: joblib (==1.2.0)
Requires-Dist: jsonlines (==3.1.0)
Requires-Dist: lit (==16.0.1)
Requires-Dist: loguru (==0.7.0)
Requires-Dist: markdown-it-py (==2.2.0)
Requires-Dist: markupsafe (==2.1.2)
Requires-Dist: mdurl (==0.1.2)
Requires-Dist: mpmath (==1.3.0)
Requires-Dist: multidict (==6.0.4)
Requires-Dist: networkx (==3.1)
Requires-Dist: nltk (==3.8.1)
Requires-Dist: nomic (==1.1.6)
Requires-Dist: numpy (==1.24.2)
Requires-Dist: nvidia-cublas-cu11 (==11.10.3.66)
Requires-Dist: nvidia-cuda-cupti-cu11 (==11.7.101)
Requires-Dist: nvidia-cuda-nvrtc-cu11 (==11.7.99)
Requires-Dist: nvidia-cuda-runtime-cu11 (==11.7.99)
Requires-Dist: nvidia-cudnn-cu11 (==8.5.0.96)
Requires-Dist: nvidia-cufft-cu11 (==10.9.0.58)
Requires-Dist: nvidia-curand-cu11 (==10.2.10.91)
Requires-Dist: nvidia-cusolver-cu11 (==11.4.0.1)
Requires-Dist: nvidia-cusparse-cu11 (==11.7.4.91)
Requires-Dist: nvidia-nccl-cu11 (==2.14.3)
Requires-Dist: nvidia-nvtx-cu11 (==11.7.91)
Requires-Dist: packaging (==23.1)
Requires-Dist: pandas (==2.0.0)
Requires-Dist: pillow (==9.5.0)
Requires-Dist: pyarrow (==11.0.0)
Requires-Dist: pydantic (==1.10.7)
Requires-Dist: pygments (==2.15.0)
Requires-Dist: python-dateutil (==2.8.2)
Requires-Dist: pytz (==2023.3)
Requires-Dist: pyyaml (==6.0)
Requires-Dist: regex (==2023.3.23)
Requires-Dist: requests (==2.28.2)
Requires-Dist: rich (==13.3.4)
Requires-Dist: scikit-learn (==1.2.2)
Requires-Dist: scipy (==1.10.1)
Requires-Dist: sentence-transformers (==2.2.2)
Requires-Dist: sentencepiece (==0.1.98)
Requires-Dist: six (==1.16.0)
Requires-Dist: sympy (==1.11.1)
Requires-Dist: threadpoolctl (==3.1.0)
Requires-Dist: tokenizers (==0.13.3)
Requires-Dist: torch (==2.0.0)
Requires-Dist: torchvision (==0.15.1)
Requires-Dist: tqdm (==4.65.0)
Requires-Dist: transformers (==4.27.4)
Requires-Dist: triton (==2.0.0)
Requires-Dist: typing-extensions (==4.5.0)
Requires-Dist: tzdata (==2023.3)
Requires-Dist: urllib3 (==1.26.15)
Requires-Dist: wheel (==0.40.0)
Requires-Dist: wonderwords (==2.2.0)
Requires-Dist: yarl (==1.8.2)

This is a vesy simple way to map your text data using [Altas from NOMIC](https://docs.nomic.ai/index.html) using the lib `click`. 

You have to create an account to get API_KEY NOMIC. 

Atlas enables you to:

Store, update and organize multi-million point datasets of unstructured text, images and embeddings.

Visually interact with your datasets from a web browser.

Run semantic search and vector operations over your datasets.

Use Atlas to:

    - Visualize, interact, collaborate and share large datasets of text and embeddings.
    
    - Collaboratively clean, tag and label your datasets
    
    - Build high-availability apps powered by semantic search
    
    - Understand and debug the latent space of your AI model trains

# How to use
### Installation

To install the necessary dependencies, run the following command:

```bash
python -m venv mymapenv 
source mymapenv/bin/activate
pip install --upgrade pip 
pip install text2mapviewer
```

## Supported Transformer Models from Hugging Face 

This project supports a variety of transformer models, including models from the Hugging Face Model Hub and sentence-transformers. Below are some examples:
    - Hugging Face Model: 'prajjwal1/bert-mini'
    - Hugging Face Model: 'Sahajtomar/french_semantic'  (french version for semantic search embedding) 
    - Sentence-Transformers Model: 'sentence-transformers/all-MiniLM-L6-v2' etc...

Please ensure that the model you choose is compatible with the project requirements and adjust the `--transformer_model_name` option accordingly.

## To map your text/csv  files

```bash
pip install -r requirements.txt
python main.py --transformer-model-name MODEL_NAME --cache_dir CACHE_DIR --batch-size BATCH_SIZE --file-path FILE_PATH
```
Remarque for the CACHE_DIR : you can setup it like ==> 

```bash
export TRANSFORMERS_CACHE=/path_to_your/transformers_cache
```

Give a fidback. 
