Metadata-Version: 2.4
Name: calibrate-suite
Version: 0.1.2
Summary: Comprehensive camera-lidar calibration suite with GUI and fleet management
Home-page: https://github.com/Opera5/calibrate-suite
Author: Jamih
Author-email: Jamih <ajamiu46@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Opera5/calibrate-suite
Project-URL: Repository, https://github.com/Opera5/calibrate-suite.git
Keywords: calibration,camera,lidar,computer-vision
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20.0
Requires-Dist: scipy>=1.7.0
Requires-Dist: scikit-learn>=1.0.0
Requires-Dist: Flask==2.3.3
Requires-Dist: requests==2.31.0
Requires-Dist: python-dotenv==1.0.0
Requires-Dist: pyyaml>=6.0
Provides-Extra: cv
Requires-Dist: opencv-python>=4.5.0; extra == "cv"
Provides-Extra: 3d
Requires-Dist: open3d>=0.13.0; extra == "3d"
Provides-Extra: visualization
Requires-Dist: matplotlib>=3.3.0; extra == "visualization"
Provides-Extra: gui
Requires-Dist: PyQt5>=5.15.0; extra == "gui"
Requires-Dist: pyqtgraph>=0.13.0; extra == "gui"
Provides-Extra: full
Requires-Dist: opencv-python>=4.5.0; extra == "full"
Requires-Dist: open3d>=0.13.0; extra == "full"
Requires-Dist: matplotlib>=3.3.0; extra == "full"
Requires-Dist: PyQt5>=5.15.0; extra == "full"
Requires-Dist: pyqtgraph>=0.13.0; extra == "full"
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: black>=23.0; extra == "dev"
Requires-Dist: flake8>=6.0; extra == "dev"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# Calibrate-Suite: Multi-Modal Sensor Calibration

A comprehensive calibration suite for LiDAR-Camera extrinsic calibration using AprilTag fiducials(for 2d-3d method) and a plane board(for 3d-3d method). Provides three calibration frameworks with CLI tools, REST API, and PyQt5 GUI for desktop and fleet-scale deployments(dummy test).

---

## 📋 Table of Contents

1. [Project Overview](#project-overview)
2. [Features](#features)
3. [Installation](#installation)
4. [Dependencies & Conflicts](#dependencies--conflicts)
5. [Quick Start](#quick-start)
6. [Calibration Frameworks](#calibration-frameworks)
7. [Command-Line Tools](#command-line-tools)
8. [Data Formats](#data-formats)
9. [Cautions & Best Practices](#cautions--best-practices)
10. [Troubleshooting](#troubleshooting)
11. [FAQ](#faq)
12. [Version](#version)

---

## 🎯 Project Overview

**Calibrate-Suite** provides production-ready tools for calibrating the extrinsic transformation between LiDAR and camera sensors in robotic platforms. The package includes:

- **Three calibration frameworks** targeting different sensor configurations (LiDAR-only, 3D-3D, 2D-3D (AprilTag-based))
- **Desktop GUI** (PyQt5) for interactive calibration workflows
- **REST API** (Flask) for distributed fleet management dummy illustration
- **Command-line tools** for automated batch processing
- **Multiple installation methods** supporting both pip and ROS2 environments

**Supports:** Ubuntu 20.04+, Python 3.8+, ROS2(humble)(optional)

### Hardware Setup

![Hardware Setup](src/gui/assets/hardware_setup.jpeg)

**Pattern Generator:** i used the [Calib io Pattern Generator](https://calib.io/pages/camera-calibration-pattern-generator) for the AprilGrid

---

## ✨ Features

### Core Calibration
- ✅ **LiDAR-Camera extrinsic calibration** using AprilTag fiducials for 2D-3D and plane board for 3D-3D
- ✅ **Three frameworks** for different sensor setups and use cases
- ✅ **Automatic corner normalization** in single-pass calibration(2D-3D)
- ✅ **Robust quality filtering** with per-pose metrics
- ✅ **Comprehensive error reporting** (reprojection, translational, rotational)
- ✅ **Multi-pose support** (20+ poses used)

### Desktop GUI
- ✅ **Tab-based interface** for all frameworks and tools
- ✅ **Real-time data visualization** with 3D point cloud overlay
- ✅ **Interactive parameter tuning** with live metrics
- ✅ **Web-based 3D viewer** for browser access(the dummy server)
- ✅ **PCD file browser** and upload functionality

### REST API (dummy Fleet Server)
- ✅ **Calibration management** endpoints
- ✅ **Result persistence** with YAML/JSON export
- ✅ **Metrics dashboard** for fleet monitoring
- ✅ **File upload/download** for distributed workflows
- ✅ **Browser-based interface** for web access

### Data Processing
- ✅ **Automatic data extraction** from ROS2 bags
- ✅ **Synchronized sensor capture** (camera + LiDAR)
- ✅ **Point cloud filtering** and preprocessing
- ✅ **AprilTag detection** and corner localization
- ✅ **Board segmentation** and quality assessment

### Deployment
- ✅ **pip installation** for standard Python environments
- ✅ **ROS2 colcon build** for local deployments on ROS2
- ✅ **System package compatibility** (no Qt/RViz conflicts)
- ✅ **Docker support** ready
- ✅ **Minimal dependencies** with optional extras

---

## 📦 Installation

### Prerequisites
```bash
# Python 3.8 or higher
python3 --version

# System packages (Ubuntu 22.04)
sudo apt update
```

### Installation Options

**Option 1: ROS2 + System Packages (Recommended for ROS users in development mode)**
```bash
# Install system dependencies
sudo apt install python3-opencv python3-open3d python3-pyqt5 python3-yaml

# Install in development mode
pip install -e .

# Or with GUI dependencies
pip install -e .[gui]

# Or with all features
pip install -e .[full]

# Build with ROS2
cd ~/calib_ws
colcon build --packages-select calibrate-suite
source install/setup.bash
```

**Option 2: pip - Core Only (Minimal)**
```bash
pip install calibrate-suite==0.1.0
```
**Option 3: pip - with full features**
```bash
pip install calibrate-suite==0.1.0 [full]
```
**Option 4: pip - Full Installation (Isolated venv recommended)**
```bash
python3 -m venv calib_env
source calib_env/bin/activate
pip install calibrate-suite==0.1.0

# Launch commands after installation
calibrate-suite  # GUI
fleet-server      # REST API (optional parameters: --host 0.0.0.0 --port 8080 --debug)
```

### Verification
```bash
# Check installation and verify commands are available
which calibrate-suite
which fleet-server

# Quick test of GUI
calibrate-suite --version

# Quick test of server
fleet-server --version
```

---

## 🔧 Dependencies & Conflicts

### Core Dependencies (Always Required)
| Package | Version | Purpose |
|---------|---------|---------|
| numpy | ≥1.20.0 | Numerical computing |
| scipy | ≥1.7.0 | Scientific functions |
| scikit-learn | ≥1.0.0 | Machine learning algorithms |
| Flask | 2.3.3 | REST API framework |
| requests | 2.31.0 | HTTP client |
| python-dotenv | 1.0.0 | Environment configuration |
| pyyaml | ≥6.0 | YAML parsing |

### Optional Dependencies (Feature-Specific)
| Extra | Packages | When to Use |
|-------|----------|------------|
| `[cv]` | opencv-python ≥4.5.0 | Computer vision, AprilTag detection |
| `[3d]` | open3d ≥0.13.0 | 3D point cloud processing |
| `[visualization]` | matplotlib ≥3.3.0 | Result visualization |
| `[gui]` | PyQt5 ≥5.15.0 | Desktop GUI application |
| `[full]` | All packages | Complete feature set |
| `[dev]` | pytest, black, flake8 | Development & testing |

### Known Conflicts & Solutions

**Issue: RViz snap glibc conflicts**
```
Problem: RViz snap conflicts with system glibc (__libc_pthread_init error)
Solution: Use system RViz instead of snap
  sudo apt install ros-humble-rviz2
  # OR continue without RViz (extractor still works)
```

**Issue: NumPy version conflicts**
```
Problem: Mixed system (1.21.5) and pip NumPy versions causing sklearn conflicts
Solution: Use consistent package sources
  # Remove pip scientific packages
  pip uninstall numpy scipy scikit-learn open3d opencv-python
  
  # Use system packages
  sudo apt install python3-numpy python3-scipy python3-sklearn python3-opencv python3-open3d
  
  # Reinstall calibrate-suite without dependencies
  pip install -e . --no-deps
  or 
  # Remove pip-installed numpy and opencv
pip uninstall numpy opencv-python

# Use system packages
sudo apt install python3-numpy python3-opencv
```

**Issue: OpenCV conflicts with system Qt/RViz**
```
Problem: pip opencv-python can conflict with system OpenCV
Solution: Use system package instead
  sudo apt install python3-opencv
  pip install calibrate-suite  # without [cv]
```

**Issue: PyQt5 conflicts in ROS2 environment**
```
Problem: pip PyQt5 may conflict with ROS2 system packages
Solution: Use system PyQt5
  sudo apt install python3-pyqt5
  pip install calibrate-suite  # without [gui]
```

**Issue: NumPy 2.x compatibility**
```
Problem: Some packages compiled against NumPy 1.x
Solution: Pin NumPy version if needed
  pip install 'numpy<2' calibrate-suite[full]
```

---

## 🚀 Quick Start

### GUI Application (Recommended)
```bash
# Launch desktop application
calibrate-suite

# Tabs available:
# - Home: Quick navigation
# - Recorder: Capture ROS2 bag data
# - Extractor: Extract frames and point clouds
# - Single LiDAR: LiDAR-only calibration
# - 3D-3D: Dense 3D point cloud calibration
# - 2D-3D: AprilTag-based calibration (recommended)
# - Calibrator: Advanced settings and batch processing
```

**GUI Workflow Tabs:**

| Home |
| ![Home Interface](src/gui/assets/GUI_homepage.png)|

| Single-LiDAR Record | Calibrate |
|---|---|
| ![Single LiDAR Record](src/gui/assets/single_lidar_record_page.png) | ![Single LiDAR Calibrate](src/gui/assets/single_lidar_calibrate_page.png) |

| 2D-3D Record page | 3D-3D Record page |
|---|---|
| ![2D-3D Record](src/gui/assets/2d-record-page.png) | ![3D-3D Record](src/gui/assets/3d-record-page.png) | 

| 2D-3D Calibrate page | 3D-3D Calibrate page |
|---|---|
| ![2D-3D Calibrator](src/gui/assets/2d-calibrate-page.png) | ![3D-3D Calibrator](src/gui/assets/3d-calibrate-page.png) | 

| 2D-3D Overlay page | 3D-3D Overlay page |
|---|---|
| ![2D-3D Calibrator](src/gui/assets/2d-overlay-page.png) | ![3D-3D Calibrator](src/gui/assets/3d-overlay-page.png) |

### Command-Line (Quick Calibration)
```bash
cd /home/jamih/calib_ws/src/calibrate-suite/src

#launching calibrator suite from terminal
# Method 1: Direct Python execution
python3 -m gui.main

# Method 2: Direct script execution
python3 gui/main.py

# 2D-3D Calibration (AprilTag-based, recommended)
python3 2d_3d_calibrator.py --data_root ../calib_data/multimodal_captures

# View results
cat ../calib_data/multimodal_captures/final_extrinsic.yaml
```

### Data Processing Pipeline
```bash
# 1. Extract synchronized data from ROS2 bag
python3 2d_3d_data_extractor.py \
  --data_root ../calib_data/multimodal_captures\
  --output ../calib_data/multimodal_captures \
  --num-poses 20

# 2. Run calibration
python3 2d_3d_calibrator.py --data_root ../calib_data/multimodal_captures

# 3. Visualize results
python3 2d_3d_overlay.py --data_root ../calib_data/multimodal_captures
```

---

## 📐 Calibration Frameworks

### Framework 1: Single LiDAR Calibration

**Purpose:** Point cloud self-registration and intrinsic validation

**Features:**
- Point cloud pair registration using Point-to-Plane ICP
- FPFH-RANSAC global registration for difficult alignments
- Hybrid initialization (centroid multi-start + FPFH-RANSAC)
- Multi-scale ICP pyramid (coarse → medium → fine)
- GICP final refinement for sub-millimeter accuracy
- Statistical outlier removal
- Adaptive Voxel-based downsampling
- Fitness score and RMSE metrics

**Input:**
- Source and target PCD files (`.pcd`)

**Output:**
- Transformation matrix (4×4)
- Fitness score (0-1)
- RMSE error (meters)
- Aligned point cloud visualization

![Single LiDAR Output](src/gui/assets/single_lidar_output.png)

**When to Use:**
- Sensor self-calibration and intrinsic validation
- Point cloud registration debugging
- Temporal drift assessment
- LiDAR-to-LiDAR pose estimation

**Note:**
- when you run single Lidar capturing under bag mode, Rviz might seem unstable, just click the pause/play button, it will stabilize after that

**GUI Usage:**
```
GUI → Single LiDAR Tab → Upload PCD files → Select method → Run → View metrics
```
---

### Framework 2: 3D-3D Camera-LiDAR Calibration

**Purpose:** Dense point cloud correspondence calibration

**Features:**
- 3D point cloud extraction from both camera and LiDAR
- DBSCAN clustering for board segmentation
- Centroid-based initial estimation
- Least-squares iterative refinement
- Per-pose quality metrics (centroid error, normal error, point-to-plane RMSE)
- Cloud overlay visualization

**Input:**
- Camera point clouds (depth+image-to-3D conversion)
- LiDAR point clouds (PCD files)
- Synchronized multi-pose captures

**Output:**
```yaml
final_extrinsic.yaml:
  rotation_matrix: 3×3
  translation_vector: [x, y, z]
  metrics:
    centroid_error_m: float
    normal_error_deg: float
    rmse_error_m: float
  per_pose_quality: [...]
```

![3D-3D Calibration Output](src/gui/assets/3d_3d_overlay_output.png)

**When to Use:**
- Both sensors provide dense 3D data
- High-precision applications required
- Full point cloud validation needed
- Stereo camera + LiDAR setups

**GUI Usage:**
```
GUI → 3D-3D Tab → Select data directory → Configure parameters → Run → View metrics
```

---

### Framework 3: 2D-3D Camera-LiDAR Calibration (Recommended) ⭐

**Purpose:** AprilTag fiducial-based corner correspondence calibration (RECOMMENDED)

**Features:**
- AprilTag detection in camera images (6×6 grid)
- 2D corner localization in pixel coordinates
- LiDAR corner extraction via PCA and DBSCAN
- 8-permutation search for optimal corner matching
- Automatic corner index normalization in single pass
- Quality filtering (best N poses by board quality)
- Huber loss optimization with outlier rejection
- Comprehensive error metrics per pose
- Reprojection and translational error reporting
- Convergence guarantees with verification pass

**Input:**
- 20+ synchronized multi-pose captures
- Camera images (arbitrary resolution, e.g., 1920×1440)
- Point cloud scans (E.g., Velodyne, RoarIndar, etc.)
- AprilTag board (8×11 grid, any tag size or grid)

```yaml
final_extrinsic.yaml:
  method: camera_pose_iterative_quality_filtered
  rotation_matrix: 3×3
  translation_vector: [x, y, z]
  euler_angles_deg: [roll, pitch, yaw]
  reprojection_error_metrics: {...}
  fitness_scores: {...}
  quality_distribution: {...}
  calibration_info: {...}
```

![2D-3D Calibration Output](src/gui/assets/2d_3d_overlay_output.jpg)**Output:**


**When to Use:**
- AprilTag board is available
- Corner-based correspondences required
- Multi-pose dataset available (20+ poses)
- Need comprehensive error metrics
- Want automatic normalization and single-pass completion
- Standard camera+LiDAR robotics applications

**GUI Usage:**
```
GUI → 2D-3D Tab → Select data directory → Configure quality filter → Run → View metrics
```

---

## 💻 Command-Line Tools(without GUI)

### Data Extraction Tools
```bash
# Synchronized extraction from ROS2 bag for 3d-3d mode
cd src
python3 3d_3d_data_extractor.py \ 
  --bag path-to-bag \ 
  --output ../calib_data/captures \ 
  --num-poses 20 \ 
  --sync-tolerance 100

# Extract camera images only for 2D-3D mode
python3 2d_3d_data_extractor.py \ 
  --bag path-to-bag \ 
  --type camera \ 
  --output ../calib_data/camera_images

# Extract LiDAR point clouds only for 2D-3D mode
python3 2d_3d_data_extractor.py \ 
  --bag path-to-bag \ 
  --type lidar \ 
  --output ../calib_data/lidar_clouds

# single lidar data extraction
python3 single_data_extractor.py \ 
  --bag path-to-bag \
```

### Calibration Tools
```bash
# 2D-3D calibration (AprilTag-based, recommended)
python3 2d_3d_calibrator.py \ 
  --data_root ../calib_data/multimodal_captures \  
  --quality-filter 0.05 \ 
  --output final_extrinsic.yaml

# 3D-3D calibration (Dense point cloud)
python3 3d_3d_calibrator.py \ 
  --camera_pcds ../calib_data/camera_pcd \ 
  --lidar_pcds ../calib_data/lidar_pcd \ 
  --output final_extrinsic.yaml

# Single LiDAR registration
python3 single_lidar_calib.py \ 
  --source source.pcd \ 
  --target target.pcd \ 
  --output registration.yaml
```

### Visualization Tools
```bash
# Generate 3d_3d mode alignment overlays
python3 3d_3d_overlay.py \ 
  --data_root ../calib_data/multimodal_captures \ 
  --output alignment_viz/

# Overlay AprilTag detection for 2d_3d
python3 2d_3d_overlay.py \ 
  --data_root ../calib_data/multimodal_captures \ 
  --pose 1

# generate/Verify corner detection in lidar and camera
python3 camtag_det.py \ 
  --data_root ../calib_data/multimodal_captures

python3 lidtag_det.py \ 
--data_root ../calib_data/multimodal_captures
```
### To use the package camera calibrator(if you're using AprilTag Grid)
```bash
# After pip install calibrate-suite
calibrate-camera --data_root /path/to/data

#For PyPI Users (after adding entry point)
pip install calibrate-suite[cv]  # Need OpenCV
calibrate-camera --data_root ./my_data

#if you already have your camera intrinsics, 
#make sure to select them in the GUI before running any tab execution 
#as it has hardcoded intrinsics params(though if you capture with the GUI, 
#the system saves camera_intrinsics.yaml once, but verify path is right in GUI)
```
---

## 📊 Data Formats

### Directory Structure (Package-Created)

When using calibrate-suite, the following directory structure is created(note: they can be changed as per your wish in the GUI or through terminal by passing --data-root flag with folder path):

```
calib_data/
├── multimodal_captures/              # Main calibration dataset
│   ├── camera_intrinsics.yaml        # Camera K, D matrices
│   ├── final_extrinsic.yaml          # Output extrinsic transformation
│   ├── calibration_metrics.json      # Per-pose metrics
│   ├── pose_1/
│   │   ├── frame.jpg                 # Original camera image
│   │   ├── anot_image.jpg            # Annotated with corner detections
│   │   ├── board_plane.pcd           # Filtered/segmented LiDAR cloud
│   │   ├── detections.yaml           # Camera and LiDAR corner detections
│   │   └── metrics.yaml              # Per-pose quality metrics
│   ├── pose_2/
│   │   └── [same structure]
│   └── alignment_viz/                # Visualization outputs
│   |   ├── pose_1_overlay.jpg        # Corner reprojection overlay
│   |   ├── pose_1_cloud.jpg          # Point cloud projection
│   |   └── ...
|----

```

### Camera Intrinsics Format

All calibrate-suite scripts use a standardized `camera_intrinsics.yaml` format. If you have your own camera intrinsics, use this exact format:

```yaml
# Standard format supported across all calibrate-suite scripts
camera_matrix: [[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]]  # 3x3 intrinsic matrix
dist_coeffs: [k1, k2, p1, p2, k3]                                # Distortion coefficients
image_size: [width, height]                                      # Image dimensions in pixels
distortion_model: "plumb_bob"                                     # OpenCV distortion model
camera_name: "camera_frame"                                       # Camera frame ID (optional)

# Example with real values:
camera_matrix: [[1364.0, 0.0, 944.66], [0.0, 1363.42, 570.16], [0.0, 0.0, 1.0]]
dist_coeffs: [0.1, -0.2, 0.001, 0.002, 0.05]
image_size: [1920, 1080]
distortion_model: "plumb_bob"
camera_name: "camera_optical_frame"
```

**Important Notes:**
- File must be named exactly `camera_intrinsics.yaml`
- Place in your data root directory (e.g., `calib_data/multimodal_captures/camera_intrinsics.yaml`)
- `camera_matrix` must be a 3×3 nested list (not a flat array)
- `dist_coeffs` must be a flat list of 5 coefficients [k1, k2, p1, p2, k3]
- All extractors, calibrators, and visualization scripts use this format

### Board Geometry Configuration
```yaml
board_type: "AprilTag"
grid_dimensions: [8, 11]              # 8×11 tag grid, it can be as per your board
tag_size_m: 0.04                     # Individual tag width
board_width_m: 0.609                 # Total board width
board_height_m: 0.457                # Total board height
tag_spacing_m: 0.005                 # Gap between tags (if any)
```

---

## ⚠️ Cautions & Best Practices

### Data Collection Best Practices

✅ **DO:**
- Capture 20+ poses with board at varying angles (0°-60° from camera normal)
- Ensure consistent, even lighting throughout capture sequence
- Keep AprilTag board planar and undamaged
- Verify sensor time synchronization (<100ms offset)
- Calibrate camera intrinsics separately beforehand(if not sure of its intrinsics)
- Document environmental conditions (lighting, reflectance, temperature)
- Perform test calibration with subset before full dataset

✗ **DON'T:**
- Use fewer than 15 poses (insufficient statistical sampling)(might be less though)
- Allow board to contact sensor brackets or cables (occlusion/deformation)
- Capture images with severe motion blur
- Mix board orientations or rotate frame mid-capture
- Have LiDAR points sparse or with extreme reflectance outliers
- Assume intrinsic parameters are accurate without verification
- Recalibrate unnecessarily (extrinsics typically remain stable)

### Quality Assessment

**Warning Signs:**
- High standard deviation in error distribution (outliers present)
- Asymmetric error distribution (systematic bias exists)
- Rotational error > 60° (check mechanical alignment)
- All 20 poses accepted without filtering (quality filter too lenient)
- Non-identity corner permutations after normalization (check board orientation)

### AprilTag Board Setup

The calibrate-suite works with **any AprilTag grid configuration**, not limited to 8×11:
- Grid size: Configurable (e.g., 4×4, 5×5, 8×8)
- Tag size: Any size (adjust `config/board_geometry.yaml`)
- Board material: Any flat surface with AprilTags
- Spacing: Regular grid required for PCA-based board frame

**Board must be:**
- Planar (flat, no warping)
- Rigidly mounted (no vibration)
- Visible to both camera and LiDAR simultaneously
- Not occluded or partially obscured

---

## 🔧 Troubleshooting

| Symptom               | Probable Cause                  | Solution             |
|---------               |---                              |---                  |
| **AprilTag not detected** | Poor lighting or high exposure time | Improve illumination, reduce exposure, check camera focus |
| **Few LiDAR corners detected (<2)** | Low point cloud density or reflectance | Move board closer, ensure LiDAR line-of-sight, check surface reflectance |
| **High reprojection error (>100px)** | Time sync offset or incorrect intrinsics | Verify camera sync (<100ms), recalibrate camera intrinsics |
| **Rotation error >90°** | Multiple outlier poses not filtered | Increase quality_filter parameter (0.1m), manually review poses |
| **Calibration fails to converge** | Dominant outliers or bad permutation search | Reduce huber_delta (1.5px), check board orientation consistency |
| **Corner indices not normalized** | Permutation search failed to find match | Verify board geometry in config, check corner detection quality |
| **All poses marked "poor"** | Board completely out of focus or occluded | Check camera focus, ensure board visibility, verify synchronization |
| **Memory error during processing** | Large point clouds not downsampled | Reduce voxel size or enable downsampling in config |

### Common Command Errors

```bash
# Error: "ModuleNotFoundError: No module named 'cv2'"
# Solution: Install OpenCV
sudo apt install python3-opencv
# OR
pip install calibrate-suite[cv]

# Error: "No module named 'open3d'"
# Solution: Install Open3D
sudo apt install python3-open3d
# OR
pip install calibrate-suite[3d]

# Error: "PyQt5 import failed" (GUI)
# Solution: Install PyQt5
sudo apt install python3-pyqt5
# OR
pip install calibrate-suite[gui]

# Error: "YAML parsing failed"
# Solution: Verify YAML format and reinstall
sudo apt install python3-yaml
pip install 'pyyaml>=6.0'
```

### REST API Server (Fleet Management)

> **Note:** The fleet server is a **dummy reference implementation** for demonstration purposes. It provides basic REST endpoints for uploading calibration results from multiple robots. You can test the upload functionality from the GUI by pointing it to your own server URL.

```bash

#launching through terminal
cd /home/jamih/calib_ws/src/calibrate-suite/src
# Method 1: Using entry point (if installed)
fleet-server

# Method 2: With custom host/port
fleet-server --host 0.0.0.0 --port 8080 --debug

# Method 3: Direct Python execution
python3 -m fleet_server.app

# Method 4: Direct script execution
python3 fleet_server/app.py


# Optional parameters:
# fleet-server --host 0.0.0.0 --port 8080 --debug

# Access dashboard: http://localhost:5000
# API endpoints: http://localhost:5000/api/...
```

**Testing from GUI:**
1. In the calibrate-suite GUI, go to the Fleet tab
2. Enter your server URL (e.g., `http://localhost:5000` or your custom server address)
3. Upload calibration results to your own fleet management backend

![Fleet Upload Interface](src/gui/assets/2d-or-3d-fleet-upload.png)

---

### Performance Optimization

If processing is slow:
```bash
# Reduce point cloud density
python3 2d_3d_calibrator.py \
  --data_root ../calib_data/multimodal_captures \
  --voxel-size 0.01              # Increase for faster processing (default 0.005)
  --max-correspondence-dist 0.2  # Increase search radius

# Reduce number of poses
python3 2d_3d_calib.py \
  --data_root ../calib_data/multimodal_captures \
  --max-poses 15                 # Use best 15 poses instead of 20
```

---

## ❓ FAQ

**Q: Which calibration framework should I use?**  
A: Framework 3 (2D-3D AprilTag-based) is recommended for standard camera+LiDAR setups. Use Framework 2 for stereo+LiDAR or dense depth+LiDAR. Use Framework 1 for LiDAR self-calibration only.

**Q: Why do I need 20 poses minimum?**  
A: Statistical robustness and noise reduction. Fewer poses provide insufficient sampling of the transformation space and make results sensitive to outliers.

**Q: Can I use a different board configuration?**  
A: Yes. Modify `config/board_geometry.yaml` with your grid size and tag dimensions. The corner detection adapts to any regular AprilTag grid.

**Q: How do I integrate the calibration into my ROS2 pipeline?**  
A: Load `final_extrinsic.yaml` into your tf2 broadcaster or calibration node. See ROS2 documentation for tf2 static transforms.

**Q: What if I have a different LiDAR type?**  
A: The point cloud processing is format-agnostic. As long as you have synchronized PCD files, the calibration works with any LiDAR (Velodyne, RoarIndar, OS1, etc.).

**Q: Can I run this in Docker?**  
A: Yes. The package is Docker-ready. Dockerfile and compose configuration are available in the repository.

**Q: How often should I recalibrate?**  
A: Extrinsics typically remain stable for months. Recalibrate only if: (1) sensors were physically moved, (2) mechanical stress/vibration occurred, or (3) recalibration validation fails.

**Q: Does the package support real-time calibration?**  
A: Current implementation is batch-based. Real-time streaming calibration is planned for future releases.

**Q: Can I export calibration for other tools?**  
A: Yes. The output is standard YAML with rotation matrix and translation vector. Compatible with ROS2 tf2, OpenCV, and most robotics frameworks.

**Q: What is the typical calibration accuracy?**  
A: Mean reprojection error typically 30-50 px depending on sensor quality and data collection. Rotational error < 5° for rigid platforms.

**Q: How do I handle multiple sensors on the same platform?**  
A: Run separate calibrations for each camera+LiDAR pair. Combine results using platform geometry (e.g., manual mounting offsets).

---

## 📄 Version

**Package Version:** 0.1.2   
**License:** MIT
