Metadata-Version: 2.4
Name: orderpulse
Version: 0.2.58
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: MacOS
Summary: High-performance Rust-powered Python library for reading NSE binary order and trade feed data.
Author: RmoneyQuant
License: Other/Proprietary License
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Changelog, https://github.com/RmoneyQuant/OrderPulse/releases
Project-URL: Documentation, https://github.com/RmoneyQuant/OrderPulse#readme
Project-URL: Homepage, https://github.com/RmoneyQuant/OrderPulse
Project-URL: Issues, https://github.com/RmoneyQuant/OrderPulse/issues
Project-URL: Repository, https://github.com/RmoneyQuant/OrderPulse
Project-URL: Source, https://github.com/RmoneyQuant/OrderPulse

# orderpulse / fastreader

[![PyPI](https://img.shields.io/pypi/v/orderpulse.svg)](https://pypi.org/project/orderpulse/)
[![Python](https://img.shields.io/pypi/pyversions/orderpulse.svg)](https://pypi.org/project/orderpulse/)
[![Platform](https://img.shields.io/badge/platform-Linux%20%7C%20Windows%20%7C%20macOS-blue)](https://pypi.org/project/orderpulse/)
[![Rust](https://img.shields.io/badge/Rust-PyO3-orange)](https://pyo3.rs/)
[![Build](https://img.shields.io/badge/build-prebuilt%20wheels-success)](https://github.com/RmoneyQuant/OrderPulse/actions)
[![License](https://img.shields.io/badge/license-Proprietary-lightgrey)](#license)

`orderpulse` is a high-performance Python library powered by Rust for reading, streaming, enriching, and analysing NSE binary order/trade feed files.

The Rust extension module is exposed to Python as:

```python
import fastreader
```

It is designed for market-data engineers, quantitative developers, and research teams who need fast access to raw NSE CM/FO binary feeds, token-level metadata enrichment, and order book snapshots directly from Python.

---

## Installation

Install directly from PyPI:

```bash
pip install orderpulse
```

Verify installation:

```python
import fastreader

print("fastreader imported successfully")
print(fastreader.__file__)
```

`orderpulse` is distributed with prebuilt binary wheels. For supported Python and platform combinations, users do **not** need to install Rust, Cargo, maturin, or local compiler tools.

---

## Why orderpulse?

Raw NSE binary order and trade feeds are large, low-level, and expensive to process in pure Python. `orderpulse` keeps the performance-critical parsing and order book logic in Rust while exposing a clean Python interface for research, analytics, and production workflows.

---

## Project links

* PyPI: https://pypi.org/project/orderpulse/
* Homepage: https://github.com/RmoneyQuant/OrderPulse
* Source: https://github.com/RmoneyQuant/OrderPulse/tree/main/src
* Repository: https://github.com/RmoneyQuant/OrderPulse
* Documentation: https://github.com/RmoneyQuant/OrderPulse#readme
* Issues: https://github.com/RmoneyQuant/OrderPulse/issues
* Changelog: https://github.com/RmoneyQuant/OrderPulse/releases

---

## Core features

| Feature                   | Description                                                                            |
| ------------------------- | -------------------------------------------------------------------------------------- |
| Rust-backed binary parser | Parses NSE order/trade binary feed packets with high performance.                      |
| Python API                | Exposes clean Python classes, dictionaries, and helper methods.                        |
| Streaming reader          | Reads large binary feed files message by message.                                      |
| Cache reader              | Loads complete files into memory for repeated access.                                  |
| Symbol enrichment         | Maps token values to symbol, expiry, strike, option type, lot size, and contract name. |
| Order book builder        | Builds bid/ask depth from decoded order and trade messages.                            |
| Snapshot export           | Exports top-level order book snapshots as CSV-compatible rows.                         |
| Path builder              | Creates standard NSE feed file paths from segment, stream id, and date.                |
| Prebuilt wheels           | Allows users to install with `pip install orderpulse` without local Rust setup.        |

---

## Quick start

```python
from fastreader import StreamingBinaryLoader, SymbolMaster, OrderbookBuilder

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"

# 1. Load contract metadata
sm = SymbolMaster()
contract_count = sm.load(CONTRACT_FILE)
print("Contracts loaded:", contract_count)

# 2. Open binary stream
loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)
loader.attach_symbol_master(sm)

# 3. Build order book
builder = OrderbookBuilder()
processed = builder.build_from_source(loader, limit=100000)
print("Processed messages:", processed)

# 4. Query snapshots
active_tokens = builder.get_active_tokens()
print("Active tokens:", len(active_tokens))

for token in active_tokens[:10]:
    print(sm.lookup(token))
    print(builder.get_snapshot(token, levels=5))
```

---

## 1. What this library does

`fastreader` helps you:

* Read NSE binary order and trade feed files.
* Stream messages one by one without loading the full file into memory.
* Load full files into an in-memory cache for repeated analysis.
* Convert binary messages into Python dictionaries.
* Read order messages and trade messages separately.
* Detect whether the stream has reached end-of-file.
* Enrich raw token values with symbol, strike price, option type, expiry, lot size, and contract name.
* Build an order book from decoded messages.
* Query best bid, best ask, spread, full depth, and top-level snapshot.
* Build standard NSE feed file paths programmatically.

---

## 2. Architecture overview

The library has five main user-facing classes:

| Class                   | Purpose                                                                                                          |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `MessageCacheReader`    | Loads the complete binary feed file into memory and lets the user fetch all, order-only, or trade-only messages. |
| `StreamingBinaryLoader` | Opens a binary feed file and reads messages one by one. Best for large files.                                    |
| `SymbolMaster`          | Loads contract master CSV and maps token to symbol metadata.                                                     |
| `OrderbookBuilder`      | Builds and queries an order book from cached or streamed messages.                                               |
| `FeedPathBuilder`       | Builds standard NSE feed file paths from segment, stream id, and date.                                           |

Internally, Rust parses raw binary packets into `OrderPacket` and `TradePacket`, wraps them as `Message::Order` or `Message::Trade`, and exposes clean Python dictionaries for end users.

---

## 3. Example files used in this README

The examples below use these paths:

```python
FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"
```

Practical note:

* A date folder can exist but still be empty on some systems.
* Always verify the contract CSV file exists before calling `sm.load(...)`.
* If your file uses standard naming (`NSE_FO_contract_DDMMYYYY.csv` or `NSE_CM_contract_DDMMYYYY.csv`), you can use `load_for_date(...)`.
* If your file has a custom name, for example `cm_contract_stream_info.csv`, use `load(...)` with the exact full path.

> Note: `SymbolMaster.load()` supports contract master parsing for token metadata lookup. Standard contract master files should include fields such as token id, ticker symbol, expiry, strike, option type, stock name, and lot size. For custom contract files, use `load(...)` with the exact full path.

---

## 4. Installation

For normal users:

```bash
pip install orderpulse
```

After installing the package, import the Python extension like this:

```python
from fastreader import (
    MessageCacheReader,
    StreamingBinaryLoader,
    OrderbookBuilder,
    SymbolMaster,
    FeedPathBuilder,
)
```

Verify import:

```python
import fastreader
print(fastreader)
```

Expected output example:

```text
<module 'fastreader' from '.../fastreader...so'>
```

If you are developing locally with Rust and PyO3, a typical development installation is:

```bash
maturin develop --release
```

---

## 5. Message dictionary format

When a message is returned to Python, it is returned as a dictionary.

### Order message dictionary

```python
{
    "message_kind": "order",
    "seq_no": 123,
    "msg_len": 38,
    "stream_id": 2,
    "msg_type": "N",
    "exch_ts": 1767000000000000000,
    "local_ts": 1767000000000001000,
    "order_id": 987654321,
    "token": 12345,
    "order_type": "B",
    "price": 250000,
    "quantity": 75,
    "flags": False,
    "token_symbol": None,
    "strike_price": None,
    "option_type": None,
}
```

Important fields:

| Field                                         | Meaning                                                        |
| --------------------------------------------- | -------------------------------------------------------------- |
| `message_kind`                                | `"order"` for order messages.                                  |
| `seq_no`                                      | Stream sequence number.                                        |
| `msg_type`                                    | NSE message type. Usually `N`, `M`, or `X` for order messages. |
| `stream_id`                                   | Stream id from binary feed header.                             |
| `order_id`                                    | Exchange order id.                                             |
| `token`                                       | Instrument token.                                              |
| `order_type`                                  | Usually `B` for buy or `S` for sell.                           |
| `price`                                       | Raw price value from feed.                                     |
| `quantity`                                    | Order quantity.                                                |
| `flags`                                       | Missed/flag status from packet.                                |
| `token_symbol`, `strike_price`, `option_type` | Populated after symbol enrichment.                             |

### Trade message dictionary

```python
{
    "message_kind": "trade",
    "seq_no": 124,
    "msg_len": 45,
    "stream_id": 2,
    "msg_type": "T",
    "exch_ts": 1767000000000000000,
    "local_ts": 1767000000000001000,
    "buy_order_id": 111111,
    "sell_order_id": 222222,
    "token": 12345,
    "trade_price": 250050,
    "trade_quantity": 50,
    "flags": False,
    "token_symbol": None,
    "strike_price": None,
    "option_type": None,
}
```

Important fields:

| Field            | Meaning                       |
| ---------------- | ----------------------------- |
| `message_kind`   | `"trade"` for trade messages. |
| `msg_type`       | `T` for trade.                |
| `buy_order_id`   | Buy-side order id.            |
| `sell_order_id`  | Sell-side order id.           |
| `token`          | Instrument token.             |
| `trade_price`    | Executed trade price.         |
| `trade_quantity` | Executed trade quantity.      |

---

# 6. `MessageCacheReader`

`MessageCacheReader` loads the full binary file into memory. Use this when the file is not too large or when you want to query the same file multiple times.

## 6.1 Create reader

```python
from fastreader import MessageCacheReader

reader = MessageCacheReader()
print(reader)
```

Expected output:

```text
<fastreader.MessageCacheReader object at ...>
```

---

## 6.2 `load_to_cache(file_path)`

Loads all supported order and trade messages into memory.

```python
from fastreader import MessageCacheReader

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

reader = MessageCacheReader()
count = reader.load_to_cache(FEED_FILE)

print("Loaded messages:", count)
```

Expected output example:

```text
Loaded messages: 2500000
```

The exact number depends on your binary feed file.

Working:

* Opens the binary feed file.
* Reads only order and trade packets.
* Stores parsed messages inside Rust memory.
* Returns the total number of loaded messages.

---

## 6.3 `get_all_messages()`

Returns all cached messages as structured `CachedMessage` objects.

```python
messages = reader.get_all_messages()

print("Total:", len(messages))
first = messages[0]
print(first.message_kind, first.seq_no, first.msg_type)
print(first.token, first.price, first.trade_price)
```

Expected output example for an order message:

```text
Total: 2500000
order 42 N
1001 500 None
```

Expected output example for a trade message:

```text
trade 99 T
5000 None 750
```

Working:

* Converts every cached Rust message into a typed Python object with attribute access.
* Common fields are always available: `message_kind`, `seq_no`, `msg_len`, `stream_id`, `msg_type`, `exch_ts`, `local_ts`, `flags`, `token`.
* Order-only fields are optional: `order_type`, `order_id`, `price`, `quantity`.
* Trade-only fields are optional: `buy_order_id`, `sell_order_id`, `trade_price`, `trade_quantity`.

---

## 6.3.1 Direct access helpers (`messages`, `len(reader)`, `reader[index]`)

`MessageCacheReader` also supports Python-style access helpers for convenience.

```python
reader = MessageCacheReader()
reader.load_to_cache(FEED_FILE)

print(len(reader))      # same as total cached messages
print(reader[0])        # first CachedMessage object
print(reader[-1])       # last CachedMessage object

# Property form (returns full list of CachedMessage objects)
all_messages = reader.messages
print(all_messages[0].message_kind, all_messages[0].seq_no)
```

Working:

* `len(reader)` returns number of cached messages.
* `reader[index]` returns one `CachedMessage` object and supports negative index.
* `reader.messages` returns the full structured list, same output type as `get_all_messages()`.

Practical note:

* For very large files, prefer `reader[index]` when you only need a few messages.
* `reader.messages` materializes all cached objects and can use significant memory.

### If you see `AttributeError: 'builtins.MessageCacheReader' object has no attribute 'messages'`

This usually means your notebook or kernel is using an older extension build.

```python
import sys, fastreader
from fastreader import MessageCacheReader

reader = MessageCacheReader()
print("python:", sys.executable)
print("fastreader:", fastreader.__file__)
print("has messages:", hasattr(reader, "messages"))
print([x for x in dir(reader) if x in ("messages", "__getitem__", "__len__", "get_all_messages")])
```

If `has messages` is `False`:

1. Rebuild/install extension in the same environment:

   ```bash
   maturin develop --release
   ```

2. Restart the notebook kernel.

3. Re-run import and setup cells.

---

## 6.4 `get_order_message()`

Returns only order messages as Python dictionaries.

```python
orders = reader.get_order_message()

print("Order count:", len(orders))
print(orders[0])
```

Expected output example:

```text
Order count: 1800000
{
    'message_kind': 'order',
    'seq_no': 42,
    'msg_len': 10,
    'stream_id': 2,
    'msg_type': 'N',
    'exch_ts': 100000,
    'local_ts': 200000,
    'order_id': 55,
    'token': 1001,
    'order_type': 'B',
    'price': 500,
    'quantity': 100,
    'flags': False,
    'token_symbol': None,
    'strike_price': None,
    'option_type': None
}
```

Working:

* Filters cached messages where `message_kind` is order.
* Converts each order packet into a Python dictionary.
* Good for pandas conversion and downstream analysis.

Example with pandas:

```python
import pandas as pd

orders_df = pd.DataFrame(reader.get_order_message())
print(orders_df.head())
```

Expected output example:

```text
  message_kind  seq_no  msg_len  stream_id msg_type  ...  quantity  flags token_symbol strike_price option_type
0        order      42       10          2        N  ...       100  False         None         None        None
```

---

## 6.5 `get_trade_message()`

Returns only trade messages as Python dictionaries.

```python
trades = reader.get_trade_message()

print("Trade count:", len(trades))
print(trades[0])
```

Expected output example:

```text
Trade count: 700000
{
    'message_kind': 'trade',
    'seq_no': 99,
    'msg_len': 10,
    'stream_id': 2,
    'msg_type': 'T',
    'exch_ts': 300000,
    'local_ts': 400000,
    'buy_order_id': 10,
    'sell_order_id': 20,
    'token': 5000,
    'trade_price': 750,
    'trade_quantity': 30,
    'flags': True,
    'token_symbol': None,
    'strike_price': None,
    'option_type': None
}
```

Working:

* Filters cached messages where `message_kind` is trade.
* Converts each trade packet into a Python dictionary.

---

## 6.6 `get_all_trade_message()`

Alias for `get_trade_message()`.

```python
trades_1 = reader.get_trade_message()
trades_2 = reader.get_all_trade_message()

print(len(trades_1) == len(trades_2))
```

Expected output:

```text
True
```

Working:

* Internally calls the same trade-message extraction logic.
* Kept for user convenience.

---

## 6.7 `get_cache_summary()`

Returns summary information about the loaded cache.

```python
summary = reader.get_cache_summary()
print(summary)
```

Expected output example:

```python
{
    'file_source': '/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin',
    'total_messages': 2500000,
    'total_orders': 1800000,
    'total_trades': 700000,
    'memory_usage_bytes': 120000000
}
```

Working:

* Counts total cached messages.
* Counts order messages.
* Counts trade messages.
* Estimates Rust-side memory usage.

---

# 7. `StreamingBinaryLoader`

`StreamingBinaryLoader` reads messages one by one from the binary file. Use this for large files where loading everything into memory is not ideal.

## 7.1 Create stream loader

```python
from fastreader import StreamingBinaryLoader

loader = StreamingBinaryLoader()
```

---

## 7.2 `open_stream(file_path, count_messages=True)`

Opens a binary file for streaming.

```python
from fastreader import StreamingBinaryLoader

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

loader = StreamingBinaryLoader()
total = loader.open_stream(FEED_FILE, count_messages=True)

print("Total messages:", total)
```

Expected output example:

```text
Total messages: 2500000
```

For faster opening on very large files, skip counting:

```python
loader = StreamingBinaryLoader()
total = loader.open_stream(FEED_FILE, count_messages=False)

print(total)
```

Expected output:

```text
0
```

Working:

* Opens the binary file.
* Validates the first message header.
* Optionally scans the full file to count messages.
* Sets the stream cursor to the beginning.

---

## 7.3 `get_next_msg()`

Reads the next message from the stream and returns a Python dictionary. Returns `None` when end-of-file is reached.

```python
msg = loader.get_next_msg()
print(msg)
```

Expected output example:

```python
{
    'message_kind': 'order',
    'seq_no': 42,
    'msg_len': 10,
    'stream_id': 2,
    'msg_type': 'N',
    'exch_ts': 100000,
    'local_ts': 200000,
    'order_id': 55,
    'token': 1001,
    'order_type': 'B',
    'price': 500,
    'quantity': 100,
    'flags': False,
    'token_symbol': None,
    'strike_price': None,
    'option_type': None
}
```

Read first five messages:

```python
for i in range(5):
    msg = loader.get_next_msg()
    if msg is None:
        print("End of file")
        break
    print(i, msg)
```

Expected output example:

```text
0 {'message_kind': 'order', 'seq_no': 1, ...}
1 {'message_kind': 'order', 'seq_no': 2, ...}
2 {'message_kind': 'trade', 'seq_no': 3, ...}
3 {'message_kind': 'order', 'seq_no': 4, ...}
4 {'message_kind': 'trade', 'seq_no': 5, ...}
```

Working:

* Reads one binary packet from the current file cursor.
* Converts it into a Python dictionary.
* Advances the cursor to the next message.
* Returns `None` after the final message.

---

## 7.4 `is_end_of_msg()`

Checks whether the stream is at the end of messages.

```python
while not loader.is_end_of_msg():
    msg = loader.get_next_msg()
    print(msg)

print("Completed")
```

Expected output example:

```text
{'message_kind': 'order', 'seq_no': 1, ...}
{'message_kind': 'trade', 'seq_no': 2, ...}
Completed
```

Working:

* Looks ahead to check whether another message exists.
* Restores the file cursor to its original position.
* Does not consume or skip any message.
* Returns `True` when no more messages are available.

Recommended safe streaming pattern:

```python
loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)

while True:
    msg = loader.get_next_msg()
    if msg is None:
        break

    # process message here
    print(msg["message_kind"], msg["token"])
```

---

## 7.5 `reset_cursor()`

Moves the stream cursor back to the beginning of the file.

```python
first_msg = loader.get_next_msg()
print("First read:", first_msg)

loader.reset_cursor()

again_first_msg = loader.get_next_msg()
print("After reset:", again_first_msg)
```

Expected output example:

```text
First read: {'message_kind': 'order', 'seq_no': 1, ...}
After reset: {'message_kind': 'order', 'seq_no': 1, ...}
```

Working:

* Seeks the file cursor to byte position `0`.
* Lets the user re-read the same file from the beginning.

---

## 7.6 `attach_symbol_master(master)`

Attaches a loaded `SymbolMaster` to the stream loader. After this, every `get_next_msg()` call automatically fills symbol metadata.

```python
from fastreader import StreamingBinaryLoader, SymbolMaster

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"

sm = SymbolMaster()
loaded = sm.load(CONTRACT_FILE)
print("Contracts loaded:", loaded)

loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)
loader.attach_symbol_master(sm)

msg = loader.get_next_msg()
print(msg)
```

Expected output example:

```python
{
    'message_kind': 'order',
    'seq_no': 42,
    'stream_id': 2,
    'msg_type': 'N',
    'token': 12345,
    'token_symbol': 'RELIANCE',
    'strike_price': -1,
    'option_type': 'XX',
    'expiry': '29-Dec-2025',
    'lot_size': 1,
    'name': 'RELIANCE'
}
```

Working:

* Stores the token-to-contract map inside the stream loader.
* Every streamed message is enriched automatically when its token exists in the symbol master.

---

## 7.7 `detach_symbol_master()`

Removes the attached symbol master.

```python
loader.detach_symbol_master()
msg = loader.get_next_msg()
print(msg["token_symbol"], msg["strike_price"], msg["option_type"])
```

Expected output:

```text
None None None
```

Working:

* Clears the attached metadata map.
* Future streamed messages return raw token fields only.

---

# 8. `SymbolMaster`

`SymbolMaster` loads the contract master CSV and provides fast token lookup and message enrichment.

## 8.1 Create symbol master

```python
from fastreader import SymbolMaster

sm = SymbolMaster()
print(sm)
print(len(sm))
```

Expected output:

```text
SymbolMaster(contracts=0)
0
```

---

## 8.2 `load(csv_path)`

Loads contract metadata from an explicit CSV file path.

```python
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"

sm = SymbolMaster()
count = sm.load(CONTRACT_FILE)

print("Loaded contracts:", count)
print(sm)
print("Length:", len(sm))
```

Safer path-discovery example, recommended for notebook use:

```python
from pathlib import Path
from fastreader import SymbolMaster

sm = SymbolMaster()

# 1. Try exact expected paths first.
candidates = [
    Path("/nas/50.30/CONTRACT/27_05_2026/NSE_FO_contract_27052026.csv"),
    Path("/home/pratima/CONTRACT/27_05_2026/NSE_FO_contract_27052026.csv"),
    Path.cwd() / "NSE_FO_contract_27052026.csv",
]

csv_path = next((str(p) for p in candidates if p.exists()), None)

# 2. Fallback: pick latest available FO contract CSV from NAS.
if csv_path is None:
    fo_files = sorted(Path("/nas/50.30/CONTRACT").glob("*/NSE_FO_contract_*.csv"))
    if fo_files:
        csv_path = str(fo_files[-1])
        print("Using latest available FO CSV:", csv_path)
    else:
        print("No NSE_FO contract CSV found under /nas/50.30/CONTRACT")

if csv_path is not None:
    count = sm.load(csv_path)
    print("Loaded contracts:", count)
```

Expected output example:

```text
Loaded contracts: 50000
SymbolMaster(contracts=50000)
Length: 50000
```

Working:

* Opens the CSV file.
* Reads contract metadata columns.
* Converts expiry values into readable dates where supported.
* Converts strike values where required.
* Stores mapping as `token -> contract metadata`.

---

## 8.3 `load_for_date(segment, day, month, year, base_path=None)`

Builds the standard contract master path automatically and loads it.

```python
sm = SymbolMaster()
count = sm.load_for_date("NSE_CM", day=10, month=10, year=2025, base_path="/nas/50.30")

print(count)
```

Expected path internally:

```text
/nas/50.30/CONTRACT/10_10_2025/NSE_CM_contract_10102025.csv
```

Expected output example:

```text
50000
```

Supported segment values:

```text
NSE_FO, FO, NSE_CM, CM
```

Working:

* Normalizes segment to `FO` or `CM`.
* Builds path using date and base path.
* Calls `load()` internally.

Important note for custom CSV paths:

```python
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"
sm.load(CONTRACT_FILE)
```

Use `load()` when your CSV filename is custom. Use `load_for_date()` only when your filename follows the library pattern:

```text
/nas/50.30/CONTRACT/DD_MM_YYYY/NSE_CM_contract_DDMMYYYY.csv
```

---

## 8.4 `lookup(token)`

Looks up one token and returns contract metadata.

```python
info = sm.lookup(12345)
print(info)
```

Expected output example when token is found:

```python
{
    'token': 12345,
    'found': True,
    'symbol': 'RELIANCE',
    'name': 'RELIANCE',
    'option_type': 'XX',
    'strike': -1,
    'expiry': '29-Dec-2025',
    'lot_size': 1
}
```

Expected output when token is not found:

```python
{
    'token': 999999999,
    'found': False,
    'symbol': None,
    'name': None,
    'option_type': None,
    'strike': None,
    'expiry': None,
    'lot_size': None
}
```

Working:

* Searches the loaded Rust hash map by token.
* Returns `found=True` and metadata if available.
* Returns `found=False` and `None` fields if unavailable.

---

## 8.5 `enrich(msg)`

Enriches a message dictionary in place.

```python
loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)

msg = loader.get_next_msg()
print("Before:", msg)

found = sm.enrich(msg)
print("Found:", found)
print("After:", msg)
```

Expected output example:

```text
Before: {'message_kind': 'order', 'token': 12345, 'token_symbol': None, 'strike_price': None, 'option_type': None, ...}
Found: True
After: {'message_kind': 'order', 'token': 12345, 'token_symbol': 'RELIANCE', 'strike_price': -1, 'option_type': 'XX', 'expiry': '29-Dec-2025', 'lot_size': 1, 'name': 'RELIANCE', ...}
```

Expected output if token is not present in contract master:

```text
Found: False
```

Working:

* Reads the `token` field from the message dictionary.
* Looks up token in loaded symbol master.
* Adds or updates:

  * `token_symbol`
  * `strike_price`
  * `option_type`
  * `expiry`
  * `lot_size`
  * `name`
* Returns `True` if enrichment happened.

---

# 9. `OrderbookBuilder`

`OrderbookBuilder` builds and queries order book depth from order/trade messages.

## 9.1 Create builder

```python
from fastreader import OrderbookBuilder

builder = OrderbookBuilder()
print(builder)
```

Expected output:

```text
<fastreader.OrderbookBuilder object at ...>
```

---

## 9.2 `apply_filter(logic_criteria=None)`

Filters which message types are processed.

```python
builder = OrderbookBuilder()
builder.apply_filter(["N", "M", "X"])
```

This means:

* Process new order messages: `N`
* Process modify order messages: `M`
* Process cancel/delete messages: `X`
* Skip trade messages: `T`

To clear the filter:

```python
builder.apply_filter(None)
```

Expected output:

```text
No direct output. Filter is applied internally.
```

Working:

* Stores allowed message types as bytes.
* During order book building, unsupported message types are skipped.

---

## 9.3 `orderbook_add_msg(msg)`

Adds one decoded message dictionary into the order book.

```python
loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)

builder = OrderbookBuilder()

msg = loader.get_next_msg()
accepted = builder.orderbook_add_msg(msg)

print("Accepted:", accepted)
```

Expected output example:

```text
Accepted: True
```

Expected output when message is skipped by filter or business rule:

```text
Accepted: False
```

Working:

* Expects one dictionary returned by `get_next_msg()`.
* Converts the Python dictionary back into Rust message format.
* Applies the order/trade update to the order book manager.
* Returns whether the message was accepted and processed.

Common error:

```python
builder.orderbook_add_msg("wrong input")
```

Expected error:

```text
TypeError: orderbook_add_msg expects one message dict from get_next_msg()
```

---

## 9.4 `build_from_list(source)`

Builds the order book from either:

1. A `MessageCacheReader`, or
2. A Python `list[dict]` of decoded messages.

### Example A: build from cache reader

```python
reader = MessageCacheReader()
reader.load_to_cache(FEED_FILE)

builder = OrderbookBuilder()
processed = builder.build_from_list(reader)

print("Processed:", processed)
```

Expected output example:

```text
Processed: 2400000
```

### Example B: build from list of dictionaries

```python
reader = MessageCacheReader()
reader.load_to_cache(FEED_FILE)

orders = reader.get_order_message()

builder = OrderbookBuilder()
processed = builder.build_from_list(orders)

print("Processed:", processed)
```

Expected output example:

```text
Processed: 1800000
```

Working:

* If source is `MessageCacheReader`, it uses internal cached Rust messages directly.
* If source is `list[dict]`, it converts each dictionary into Rust message format.
* Applies each message to the order book.
* Returns count of processed messages.

---

## 9.5 `build_from_source(source, limit=None)`

Builds the order book from either a `MessageCacheReader` or a `StreamingBinaryLoader`.

### Build from cache reader

```python
reader = MessageCacheReader()
reader.load_to_cache(FEED_FILE)

builder = OrderbookBuilder()
processed = builder.build_from_source(reader)

print(processed)
```

Expected output example:

```text
2400000
```

### Build from streaming loader with limit

```python
loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(loader, limit=10000)

print("Processed first messages:", processed)
```

Expected output example:

```text
Processed first messages: 10000
```

Working:

* If source is a cache reader, delegates to `build_from_list()`.
* If source is a streaming loader, reads from the current stream cursor.
* Stops at end-of-file or after `limit` accepted messages.

Important:

```python
processed = builder.build_from_source(loader, limit=10000)
```

The `limit` counts accepted/processed messages, not necessarily raw messages read if filters skip some messages.

---

## 9.6 `get_active_tokens()`

Returns all tokens currently present in the order book.

```python
active_tokens = builder.get_active_tokens()

print("Number of active tokens:", len(active_tokens))
print(active_tokens[:10])
```

Expected output example:

```text
Number of active tokens: 1250
[12345, 12346, 12347, 2885, 3045, 11536, 1333, 4963, 1594, 1660]
```

Working:

* Queries the order book manager.
* Returns tokens for which depth/order book state exists.

---

## 9.7 `get_snapshot(token, levels=None)`

Returns top order book levels for a token.

```python
token = builder.get_active_tokens()[0]
snapshot = builder.get_snapshot(token, levels=5)

print(snapshot)
```

Expected output example when token is found:

```python
{
    'token': 12345,
    'found': True,
    'mid_price': 250025,
    'best_bid': (250000, 100),
    'best_ask': (250050, 150),
    'spread': 50,
    'bids': [(250000, 100), (249950, 75), (249900, 50)],
    'asks': [(250050, 150), (250100, 25), (250150, 10)]
}
```

Expected output when token is not found:

```python
{
    'token': 999999999,
    'found': False,
    'mid_price': 0,
    'best_bid': None,
    'best_ask': None,
    'spread': None,
    'bids': [],
    'asks': []
}
```

Working:

* Reads top bid and ask levels from the order book.
* Calculates:

  * `best_bid`
  * `best_ask`
  * `spread = best_ask_price - best_bid_price`
  * `mid_price`
* Defaults to 5 levels if `levels` is not provided.

---

## 9.8 `get_full_depth(token)`

Returns full available depth for a token.

```python
full_depth = builder.get_full_depth(token)
print(full_depth)
```

Expected output example:

```python
{
    'token': 12345,
    'found': True,
    'best_bid': (250000, 100),
    'best_ask': (250050, 150),
    'spread': 50,
    'bids': [(250000, 100), (249950, 75), (249900, 50), ...],
    'asks': [(250050, 150), (250100, 25), (250150, 10), ...]
}
```

Expected output when token is not found:

```python
{
    'token': 999999999,
    'found': False,
    'best_bid': None,
    'best_ask': None,
    'spread': None,
    'bids': [],
    'asks': []
}
```

Working:

* Returns all available bid and ask levels for the token.
* Best for detailed order book inspection.
* Use `get_snapshot()` for faster top-N view.

---

## 9.9 `snapshot_header()`

Returns CSV header for snapshot rows.

```python
header = builder.snapshot_header()
print(header)
```

Expected output:

```text
local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,ask_qty_1,bid_price_2,bid_qty_2,ask_price_2,ask_qty_2,bid_price_3,bid_qty_3,ask_price_3,ask_qty_3,bid_price_4,bid_qty_4,ask_price_4,ask_qty_4
```

Working:

* Returns a fixed CSV header for 5-level snapshot output.

---

## 9.10 `get_snapshot_row(token, levels=None)`

Returns one token snapshot as a CSV row string.

```python
row = builder.get_snapshot_row(token, levels=5)
print(builder.snapshot_header())
print(row)
```

Expected output example:

```text
local_ts,exch_ts,mid_price,bid_price_0,bid_qty_0,ask_price_0,ask_qty_0,bid_price_1,bid_qty_1,ask_price_1,bid_qty_1,...
0,0,250025,250000,100,250050,150,249950,75,250100,25,249900,50,250150,10,0,0,0,0,0,0,0,0
```

Working:

* Fetches top levels from the order book.
* Pads missing bid/ask levels with `0,0`.
* Returns a CSV-compatible row.

Save snapshots to CSV:

```python
with open("snapshots.csv", "w") as f:
    f.write(builder.snapshot_header() + "\n")
    for token in builder.get_active_tokens():
        f.write(builder.get_snapshot_row(token, levels=5) + "\n")
```

---

# 10. `FeedPathBuilder`

`FeedPathBuilder` creates standard NSE feed file paths.

## 10.1 Create path builder

```python
from fastreader import FeedPathBuilder

path_builder = FeedPathBuilder()
print(path_builder)
```

Expected output:

```text
FeedPathBuilder()
```

---

## 10.2 `build(segment, stream_id, day, month, year, base_path=None)`

Builds a feed file path.

```python
builder = FeedPathBuilder()

path = builder.build(
    "NSE_CM",
    stream_id=2,
    day=29,
    month=12,
    year=2025,
)

print(path)
```

Expected output:

```text
/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
```

With custom base path:

```python
path = builder.build(
    "NSE_CM",
    stream_id=2,
    day=29,
    month=12,
    year=2025,
    base_path="/nas/50.30",
)

print(path)
```

Expected output:

```text
/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
```

Working:

* Accepts segment values such as `NSE_CM`, `CM`, `NSE_FO`, or `FO`.
* Validates stream id and date values.
* Returns the standard feed file path string.

---

## 10.3 `build_and_verify(segment, stream_id, day, month, year, base_path=None)`

Builds the path and checks whether the file exists.

```python
builder = FeedPathBuilder()

path = builder.build_and_verify(
    "NSE_CM",
    stream_id=2,
    day=29,
    month=12,
    year=2025,
    base_path="/nas/50.30",
)

print(path)
```

Expected output if file exists:

```text
/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
```

Expected error if file does not exist:

```text
RuntimeError: file does not exist: /nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
```

Working:

* Builds the same path as `build()`.
* Performs disk existence check.
* Raises `RuntimeError` if file is missing.

---

# 11. Complete workflow examples

## 11.1 Fast cached workflow

Use this when memory is sufficient and you want repeated access.

```python
from fastreader import MessageCacheReader, OrderbookBuilder

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

reader = MessageCacheReader()
count = reader.load_to_cache(FEED_FILE)
print("Loaded:", count)

summary = reader.get_cache_summary()
print(summary)

builder = OrderbookBuilder()
processed = builder.build_from_list(reader)
print("Processed into orderbook:", processed)

active_tokens = builder.get_active_tokens()
print("Active tokens:", len(active_tokens))

if active_tokens:
    token = active_tokens[0]
    print("Snapshot:", builder.get_snapshot(token, levels=5))
    print("Full depth:", builder.get_full_depth(token))
```

Expected output example:

```text
Loaded: 2500000
{'file_source': '/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin', 'total_messages': 2500000, 'total_orders': 1800000, 'total_trades': 700000, 'memory_usage_bytes': 120000000}
Processed into orderbook: 2400000
Active tokens: 1250
Snapshot: {'token': 12345, 'found': True, 'mid_price': 250025, ...}
Full depth: {'token': 12345, 'found': True, 'best_bid': (250000, 100), ...}
```

---

## 11.2 Memory-friendly streaming workflow

Use this for very large files.

```python
from fastreader import StreamingBinaryLoader, OrderbookBuilder

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)

builder = OrderbookBuilder()
processed = builder.build_from_source(loader, limit=100000)

print("Processed:", processed)
print("Active tokens:", len(builder.get_active_tokens()))
```

Expected output example:

```text
Processed: 100000
Active tokens: 450
```

---

## 11.3 Streaming with automatic symbol enrichment

```python
from fastreader import StreamingBinaryLoader, SymbolMaster

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"

sm = SymbolMaster()
sm.load(CONTRACT_FILE)

loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)
loader.attach_symbol_master(sm)

for _ in range(10):
    msg = loader.get_next_msg()
    if msg is None:
        break

    print(
        msg["message_kind"],
        msg["token"],
        msg.get("token_symbol"),
        msg.get("strike_price"),
        msg.get("option_type"),
    )
```

Expected output example:

```text
order 12345 RELIANCE -1 XX
order 2885 HDFCBANK -1 XX
trade 3045 SBIN -1 XX
```

---

## 11.4 Manual enrichment of cached/streamed messages

```python
from fastreader import StreamingBinaryLoader, SymbolMaster

sm = SymbolMaster()
sm.load("/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv")

loader = StreamingBinaryLoader()
loader.open_stream("/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin", count_messages=False)

msg = loader.get_next_msg()
found = sm.enrich(msg)

print("Enriched:", found)
print(msg)
```

Expected output example:

```text
Enriched: True
{'message_kind': 'order', 'token': 12345, 'token_symbol': 'RELIANCE', 'strike_price': -1, 'option_type': 'XX', 'expiry': '29-Dec-2025', 'lot_size': 1, 'name': 'RELIANCE', ...}
```

---

## 11.5 Build path and open stream

```python
from fastreader import FeedPathBuilder, StreamingBinaryLoader

path_builder = FeedPathBuilder()

feed_path = path_builder.build(
    "NSE_CM",
    stream_id=2,
    day=29,
    month=12,
    year=2025,
    base_path="/nas/50.30",
)

print(feed_path)

loader = StreamingBinaryLoader()
loader.open_stream(feed_path, count_messages=False)

msg = loader.get_next_msg()
print(msg)
```

Expected output:

```text
/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin
{'message_kind': 'order', 'seq_no': 1, ...}
```

---

## 11.6 Export top-5 snapshots to CSV

```python
from fastreader import MessageCacheReader, OrderbookBuilder

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"

reader = MessageCacheReader()
reader.load_to_cache(FEED_FILE)

builder = OrderbookBuilder()
builder.build_from_list(reader)

with open("orderbook_snapshots.csv", "w") as f:
    f.write(builder.snapshot_header() + "\n")

    for token in builder.get_active_tokens():
        f.write(builder.get_snapshot_row(token, levels=5) + "\n")

print("Saved orderbook_snapshots.csv")
```

Expected output:

```text
Saved orderbook_snapshots.csv
```

---

# 12. Error handling examples

## 12.1 Missing binary file

```python
reader = MessageCacheReader()
reader.load_to_cache("/wrong/path/file.bin")
```

Expected error:

```text
RuntimeError: No such file or directory
```

---

## 12.2 Missing contract CSV

```python
sm = SymbolMaster()
sm.load("/wrong/path/contracts.csv")
```

Expected error:

```text
RuntimeError: cannot open /wrong/path/contracts.csv: No such file or directory
```

Checklist:

* Confirm file exists with `Path(path).exists()`.
* If using `load_for_date(...)`, confirm the generated filename follows `NSE_{FO|CM}_contract_DDMMYYYY.csv`.
* If your CSV has a custom filename, switch to `sm.load(full_custom_path)`.

---

## 12.2A SymbolMaster object not created

```python
loaded = sm.load("/some/path/NSE_FO_contract_27052026.csv")
```

Expected error:

```text
NameError: name 'sm' is not defined
```

Fix:

```python
from fastreader import SymbolMaster

sm = SymbolMaster()
loaded = sm.load("/some/path/NSE_FO_contract_27052026.csv")
print(loaded)
```

---

## 12.3 Wrong input to order book

```python
builder = OrderbookBuilder()
builder.orderbook_add_msg(["not", "a", "dict"])
```

Expected error:

```text
TypeError: orderbook_add_msg expects one message dict from get_next_msg()
```

---

## 12.4 Unsupported message type in dictionary

```python
builder = OrderbookBuilder()
builder.build_from_list([{"msg_type": "Z"}])
```

Expected error:

```text
TypeError: unsupported msg_type: Z
```

---

# 13. Best practices

## Use `MessageCacheReader` when:

* File size is manageable.
* You need to repeatedly access messages.
* You want to convert orders/trades into pandas DataFrames.
* You want fast order book building from memory.

## Use `StreamingBinaryLoader` when:

* File is very large.
* You want low memory usage.
* You want to process messages one by one.
* You want to stop after a fixed limit.

## Use `SymbolMaster` when:

* You need token-to-symbol mapping.
* You want `token_symbol`, `strike_price`, `option_type`, `expiry`, `lot_size`, and `name` in messages.
* You want cleaner outputs for library users.

## Use `OrderbookBuilder` when:

* You need best bid/best ask.
* You need spread and mid price.
* You need top-5 or full depth by token.
* You need CSV-style snapshot rows.

## Use `FeedPathBuilder` when:

* Your files follow the standard `/nas/50.30` feed path pattern.
* You want to avoid hardcoding feed paths.
* You want path validation before opening files.

---

# 14. Full end-to-end example

```python
from fastreader import (
    FeedPathBuilder,
    StreamingBinaryLoader,
    SymbolMaster,
    OrderbookBuilder,
)

# 1. Build feed path
path_builder = FeedPathBuilder()
feed_path = path_builder.build(
    "NSE_CM",
    stream_id=2,
    day=29,
    month=12,
    year=2025,
    base_path="/nas/50.30",
)

# 2. Load contract metadata
contract_path = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"
sm = SymbolMaster()
contract_count = sm.load(contract_path)
print("Contracts loaded:", contract_count)

# 3. Open binary stream
loader = StreamingBinaryLoader()
loader.open_stream(feed_path, count_messages=False)
loader.attach_symbol_master(sm)

# 4. Build orderbook from first 100000 accepted messages
builder = OrderbookBuilder()
builder.apply_filter(["N", "M", "X", "T"])
processed = builder.build_from_source(loader, limit=100000)
print("Processed messages:", processed)

# 5. Query snapshots
active_tokens = builder.get_active_tokens()
print("Active tokens:", len(active_tokens))

for token in active_tokens[:5]:
    info = sm.lookup(token)
    snapshot = builder.get_snapshot(token, levels=5)

    print("Token info:", info)
    print("Snapshot:", snapshot)
```

Expected output example:

```text
Contracts loaded: 50000
Processed messages: 100000
Active tokens: 450
Token info: {'token': 12345, 'found': True, 'symbol': 'RELIANCE', 'name': 'RELIANCE', 'option_type': 'XX', 'strike': -1, 'expiry': '29-Dec-2025', 'lot_size': 1}
Snapshot: {'token': 12345, 'found': True, 'mid_price': 250025, 'best_bid': (250000, 100), 'best_ask': (250050, 150), 'spread': 50, 'bids': [...], 'asks': [...]}
```

---

# 15. Developer notes

* Binary message parsing is handled in Rust for performance.
* Python receives normal dictionaries, lists, strings, and integers.
* `get_next_msg()` returns `None` at end-of-file.
* `is_end_of_msg()` checks EOF without advancing the cursor.
* Symbol enrichment is optional.
* `attach_symbol_master()` is best for streaming workflows.
* `SymbolMaster.enrich()` is useful when you already have a message dictionary.
* `build_from_source()` is the most flexible order book API because it accepts both cached and streaming sources.
* `get_snapshot()` is best for application display.
* `get_snapshot_row()` and `snapshot_header()` are best for CSV export.

---

# 16. Minimal quick-start

```python
from fastreader import StreamingBinaryLoader, SymbolMaster, OrderbookBuilder

FEED_FILE = "/nas/50.30/NSE_CM/Feed_CM_StreamID_2_29_12_2025.bin"
CONTRACT_FILE = "/nas/50.30/CONTRACT/10_10_2025/cm_contract_stream_info.csv"

sm = SymbolMaster()
sm.load(CONTRACT_FILE)

loader = StreamingBinaryLoader()
loader.open_stream(FEED_FILE, count_messages=False)
loader.attach_symbol_master(sm)

builder = OrderbookBuilder()
builder.build_from_source(loader, limit=100000)

for token in builder.get_active_tokens()[:10]:
    print(sm.lookup(token))
    print(builder.get_snapshot(token, levels=5))
```

This is the recommended starting point for most library users.

---

# 17. Developer build and release

This section is for maintainers and developers building from source.

## 17.1 Build locally

From the package directory:

```bash
cd OrderPulse
maturin build --release
```

Example output:

```text
target/wheels/orderpulse-0.2.58-cp39-cp39-manylinux_2_34_x86_64.whl
```

## 17.2 Install local wheel

```bash
python -m pip uninstall -y orderpulse fastreader
python -m pip install --force-reinstall ../target/wheels/orderpulse-0.2.58-cp39-cp39-manylinux_2_34_x86_64.whl
```

## 17.3 Test local install

```python
import fastreader

print("fastreader imported successfully")
print(fastreader.__file__)
```
