Metadata-Version: 2.4
Name: jsonl-archive-splitter
Version: 0.1.2
Summary: CLI tool to split JSONL archive files by Jalali boundary date.
Author: Saeed
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: jdatetime

# JSONL Archive Splitter

A production-ready Python CLI tool for splitting large JSONL archive
files based on a Jalali (Persian) boundary date and configurable maximum
file size.

------------------------------------------------------------------------

## ✨ Features

-   Split JSONL files based on `publish_ts`
-   Supports Jalali (Shamsi) boundary dates
-   Automatic file rotation by maximum size
-   Line-by-line processing (memory efficient)
-   Modular architecture (CLI / Core / Utils separation)
-   Logging-based execution (no noisy prints)
-   Ready for packaging and future PyPI publishing

------------------------------------------------------------------------

## 📦 Installation

using source code (GitHub repository):

``` bash
git clone https://github.com/ngsaeed/jsonl-archive-splitter.git
cd jsonl-archive-splitter
```

using pip:

``` bash
pip install jsonl-archive-splitter
```

`requirements.txt`:

    jdatetime

------------------------------------------------------------------------

## 🚀 Usage

Run from the project root directory:

``` bash
jsonl-archive-splitter
```

Or use arguments to best fit your requirements:
``` bash
jsonl-archive-splitter --src-dir D:\\jsonl-archive-splitter\\src --before-dir D:\\jsonl-archive-splitter\\output\\part1 --after-dir D:\\jsonl-archive-splitter\\output\\part2 --boundary-date 1404-07-30 --start 1 --end 10 --max-size 400
```

Expected Source File Pattern (Default)

By default, the tool looks for files with this naming pattern:

`archive_messages_<index>.jsonl`  

Example:

`archive_messages_1.jsonl`  
`archive_messages_2.jsonl`  
`archive_messages_3.jsonl`  

you can set prefix or postfix for making the pattern that best fits to your archive. for example your archive is like `messages_<index>.txt` so:

```bash
jsonl-archive-splitter --src-prefix 'messages_' --src-postfix '.txt'
```
This will look for files:   
`messages_1.txt`  
`...`  
`messages_10.txt`  

By default, it processes only files between index 1 and 10:

`archive_messages_1.jsonl`  
`...`  
`archive_messages_10.jsonl`

But you can set it to process files between whatever indices you want. for example:

```Bash
jsonl-archive-splitter --start 201 --end 400
```

will look for following files:  
`archive_messages_201.jsonl`  
`...`  
`archive_messages_400.jsonl`  

The boundary date must be in Jalali format: `YYYY-MM-DD` for example:
```shell
jsonl-archive-splitter --boundary-date '1404-12-04'
```

------------------------------------------------------------------------

## 🧾 CLI Arguments

  Argument            Description
  ------------------- ---------------------------------------------------
  `--src-dir`         Directory containing input JSONL files (default: D:\\jsonl-archive-splitter\\src)  
  `--before-dir`      Output directory to store files for messages up to boundary date (default: D:\\jsonl-archive-splitter\\output\\part1)  
  `--after-dir`       Output directory to store files for messages after boundary date (default: D:\\jsonl-archive-splitter\\output\\part2)  
  `--boundary-date`   Jalali date in format YYYY-MM-DD (default 1404-07-30)  
  `--start`           Start file index (default: 1)  
  `--end`             End file index (default: 10)  
  `--max-size`        Max size per output file in MB (default: 400)  
  `--src-prefix`      Source files prefix (default: archive_messages_)  
  `--src-postfix`     Output files format or postfix (default: .jsonl)  
  `--out-prefix`      Output files prefix (default: archive_messages_)  

------------------------------------------------------------------------

## 🧠 How It Works

1.  Converts Jalali boundary date to Gregorian.
2.  Computes timestamp for 23:59:59 of boundary date.
3.  Iterates over JSONL files sequentially.
4.  Routes messages based on `publish_ts`.
5.  Rotates output files automatically when size limit is reached.

------------------------------------------------------------------------

## 🔮 Possible Future Improvements

-   Add unit tests
-   Add tqdm progress bar
-   Add dry-run mode

------------------------------------------------------------------------

## 📄 License

MIT License

------------------------------------------------------------------------

## 👤 Author

Saeed Naghibi -- Front-End Software Engineer \ Algorithm Specialist  
GitHub: https://github.com/ngsaeed  
LinkedIn: https://www.linkedin.com/in/ngsaeed/  
