# Project Layout Analysis - ProxySQL

> **⚠️ Important Notice**: This documentation was generated by AI and may contain inaccuracies.
> It should be used as a starting point for exploration only. Always verify critical information
> against the actual source code.
>
> **Last AI Update**: 2025-09-11
> **Status**: NON-VERIFIED
> **Maintainer**: Rene Cannao

## Executive Summary

ProxySQL is a database proxy server written in C++ providing protocol-aware proxying for MySQL and PostgreSQL databases. It implements a multi-threaded, event-driven architecture with connection pooling, query routing, and monitoring. Built with C++11/17, it uses modular design separating protocol handlers, session management, and administrative interfaces, backed by embedded SQLite3 configuration.

## Key Components

- **Dual protocols**: MySQL and PostgreSQL wire protocol implementations
- **Multi-threaded**: Separate worker threads for connections, admin, monitoring, and clustering
- **Configuration**: SQLite3 three-tier system (Disk → Memory → Runtime)
- **Performance features**: Lock-free structures, connection pooling, query caching
- **Enterprise features**: Clustering, monitoring, REST API, and Prometheus metrics

## System Architecture

```mermaid
graph TB
    subgraph "Client Layer"
        MC[MySQL Clients<br/>Port 6033]
        PC[PostgreSQL Clients<br/>Port 6033]
        AC[Admin Clients<br/>Port 6032]
    end
    
    subgraph "ProxySQL Core"
        MT[MySQL Thread Pool<br/>Protocol Handler]
        PT[PgSQL Thread Pool<br/>Protocol Handler]
        AT[Admin Thread<br/>Configuration & Stats]
        MON[Monitor Thread<br/>Health Checking]
        
        MC --> MT
        PC --> PT
        AC --> AT
    end
    
    subgraph "Core Services"
        QP[Query Processor<br/>Routing & Rewriting]
        QC[Query Cache<br/>Result Caching]
        HGM[HostGroups Manager<br/>Connection Pooling]
        AUTH[Authentication<br/>Multiple Methods]
        
        MT --> QP
        PT --> QP
        QP --> QC
        QP --> HGM
        MT --> AUTH
        PT --> AUTH
    end
    
    subgraph "Persistence Layer"
        SQL[(SQLite3<br/>Configuration DB)]
        DISK[Disk Config<br/>proxysql.cnf]
        STATS[Statistics Tables<br/>Runtime Metrics]
        
        AT --> SQL
        SQL --> DISK
        MON --> STATS
        SQL --> STATS
    end
    
    subgraph "Backend Databases"
        MDB[(MySQL/MariaDB<br/>Backends)]
        PDB[(PostgreSQL<br/>Backends)]
        
        HGM --> MDB
        HGM --> PDB
    end
    
    style MC fill:#e1f5fe
    style PC fill:#c8e6c9
    style AC fill:#fff3e0
    style SQL fill:#f3e5f5
```

## Threading Model and Data Flow

```mermaid
graph LR
    subgraph "Connection Lifecycle"
        NEW[New Connection] --> ACCEPT[Accept Thread]
        ACCEPT --> WORKER[Worker Thread<br/>Assigned]
        WORKER --> SESSION[Session Object<br/>Created]
        SESSION --> PROCESS[Query Processing]
        PROCESS --> BACKEND[Backend Connection<br/>from Pool]
        BACKEND --> EXECUTE[Execute on Backend]
        EXECUTE --> CACHE[Cache Results<br/>if eligible]
        CACHE --> RESPONSE[Return to Client]
    end
    
    subgraph "Configuration Flow"
        FILE[proxysql.cnf] --> LOAD[Load to Memory]
        LOAD --> RUNTIME[Apply to Runtime]
        RUNTIME --> PERSIST[Save to Disk]
        ADMIN[Admin Interface] --> RUNTIME
        CLUSTER[Cluster Sync] --> RUNTIME
    end
    
    subgraph "Monitoring Flow"
        CHECK[Health Check] --> STATUS[Backend Status]
        STATUS --> HOSTGROUP[Update HostGroup]
        HOSTGROUP --> ROUTING[Adjust Routing]
        STATS2[Collect Stats] --> EXPOSE[Expose Metrics]
        EXPOSE --> PROM[Prometheus/REST]
    end
    
    style NEW fill:#e8f5e8
    style FILE fill:#fff3e0
    style CHECK fill:#ffebee
```

## Directory Structure Mapping

### Code Organization
```
https://github.com/sysown/proxysql/tree/v3.0.agentics/
├── src/                        # Main entry points (4 files)
│   ├── main.cpp               # Application entry, thread initialization
│   ├── SQLite3_Server.cpp     # Configuration database
│   ├── proxy_tls.cpp          # TLS/SSL implementation
│   └── proxysql_global.cpp    # Global variables, configuration
│
├── lib/                        # Core library implementations (86 .cpp files)
│   ├── MySQL_*.cpp            # MySQL protocol & management (20+ files)
│   │   ├── MySQL_Protocol.cpp        # Wire protocol
│   │   ├── MySQL_Session.cpp         # Session handling
│   │   ├── MySQL_HostGroups_Manager.cpp # Backend management
│   │   └── MySQL_Monitor.cpp         # Health monitoring
│   │
│   ├── PgSQL_*.cpp            # PostgreSQL protocol & management (15+ files)
│   │   ├── PgSQL_Protocol.cpp        # Wire protocol v3
│   │   ├── PgSQL_Session.cpp         # Session handling
│   │   └── PgSQL_Authentication.cpp  # SASL/SCRAM
│   │
│   ├── ProxySQL_Admin*.cpp    # Administrative interface (10+ files)
│   │   ├── ProxySQL_Admin.cpp        # Admin implementation
│   │   ├── ProxySQL_Admin_Stats.cpp  # Statistics
│   │   └── ProxySQL_RESTAPI_Server.cpp # REST API
│   │
│   ├── Base_*.cpp             # Base infrastructure (5+ files)
│   ├── Query_*.cpp            # Query processing & caching
│   └── libproxysql.a          # Compiled static library (340MB+)
│
├── include/                    # Header files (89 .h files)
│   ├── MySQL_*.h              # MySQL protocol headers
│   ├── PgSQL_*.h              # PostgreSQL protocol headers
│   ├── proxysql_*.h           # Core infrastructure headers
│   └── btree*.h               # Performance data structures
│
├── deps/                       # External dependencies (23 libraries)
│   ├── mariadb-client-library/ # MySQL/MariaDB connector
│   ├── postgresql/             # PostgreSQL client library
│   ├── sqlite3/                # Embedded database
│   ├── libev/                  # Event loop library
│   ├── jemalloc/               # Memory allocator
│   ├── prometheus-cpp/         # Metrics library
│   └── ...                     # 17 more dependencies
│
└── test/                       # Test infrastructure
    ├── tap/tests/              # 220+ TAP tests
    ├── cluster/                # Cluster tests
    └── PrepStmt/               # Prepared statement tests
```

### Build System and Artifacts

```mermaid
graph TB
    subgraph "Build Targets"
        BT1[make → Standard build]
        BT2[make debug → Debug build]
        BT3[make build_deps → Build dependencies]
        BT4[make build_tap_test_debug → Test framework]
    end
    
    subgraph "Build Artifacts"
        BA1[src/proxysql → Main binary]
        BA2[lib/libproxysql.a → Static library]
        BA3[lib/obj/*.o → Object files]
        BA4[out/production/proxysql → Production binary]
        BA5[out/test/proxysql → Test binary]
    end
    
    subgraph "Dependencies Built"
        DB1[deps/mariadb-client-library/]
        DB2[deps/postgresql/]
        DB3[deps/sqlite3/]
        DB4[deps/jemalloc/]
        DB5[deps/libev/]
    end
    
    BT1 --> BA1
    BT1 --> BA2
    BT2 --> BA4
    BT3 --> DB1
    BT3 --> DB2
    BT3 --> DB3
    BT4 --> BA5
    
    style BT1 fill:#e8f5e8
    style BT2 fill:#fff3e0
    style BA1 fill:#e1f5fe
```

## Module Dependencies and Communication

```mermaid
graph LR
    subgraph "Protocol Layer"
        MYSQL[MySQL Protocol]
        PGSQL[PgSQL Protocol]
    end
    
    subgraph "Session Management"
        MSESS[MySQL Sessions]
        PSESS[PgSQL Sessions]
        BSESS[Base Session]
        
        MSESS --> BSESS
        PSESS --> BSESS
    end
    
    subgraph "Backend Management"
        MHGM[MySQL HostGroups]
        PHGM[PgSQL HostGroups]
        BHGM[Base HostGroups]
        POOL[Connection Pool]
        
        MHGM --> BHGM
        PHGM --> BHGM
        BHGM --> POOL
    end
    
    subgraph "Core Services"
        QP2[Query Processor]
        QC2[Query Cache]
        AUTH2[Authentication]
        MON2[Monitor Service]
    end
    
    subgraph "Infrastructure"
        SQLITE[SQLite3 Config]
        EVENT[libev Events]
        MEM[jemalloc Memory]
        METRICS[Prometheus Metrics]
    end
    
    MYSQL --> MSESS
    PGSQL --> PSESS
    MSESS --> QP2
    PSESS --> QP2
    QP2 --> QC2
    QP2 --> MHGM
    QP2 --> PHGM
    MON2 --> MHGM
    MON2 --> PHGM
    
    MSESS --> EVENT
    PSESS --> EVENT
    POOL --> MEM
    MON2 --> METRICS
    
    style MYSQL fill:#e1f5fe
    style PGSQL fill:#c8e6c9
    style SQLITE fill:#f3e5f5
```

## Configuration and Deployment Architecture

### Three-Tier Configuration System
```
┌─────────────────┐
│   Disk Layer    │  ← proxysql.cnf, persistent storage
├─────────────────┤
│  Memory Layer   │  ← Loaded configuration, staging area
├─────────────────┤
│ Runtime Layer   │  ← Active configuration, live system
└─────────────────┘
```

### Port Mapping
- **6033**: MySQL/PostgreSQL client connections
- **6032**: Admin interface (MySQL protocol)
- **6080**: REST API (HTTP/JSON)
- **6070**: Web UI (optional)

### Docker Deployment Scenarios
```
docker/
├── 1backend/                   # Single backend configuration
├── 5backends-replication/      # Multi-backend with replication
├── images/                     # Container image definitions
│   ├── mysql/                 # MySQL test containers
│   └── proxysql/              # ProxySQL containers
└── scenarios/                  # Testing scenarios
```

## Key Configuration and Operational Patterns

### Runtime Configuration Management
```sql
-- Admin interface SQL commands (port 6032)
INSERT INTO mysql_servers(...) VALUES(...);    -- Add backend
LOAD MYSQL SERVERS TO RUNTIME;                 -- Apply changes
SAVE MYSQL SERVERS TO DISK;                    -- Persist

-- Monitor configuration
UPDATE global_variables SET variable_value=...;
LOAD ADMIN VARIABLES TO RUNTIME;
```

### Query Routing Rules
```sql
-- Define routing rules
INSERT INTO mysql_query_rules (
    rule_id, match_pattern, destination_hostgroup
) VALUES (1, '^SELECT.*FOR UPDATE', 0);       -- Write queries to HG 0
```

### Health Monitoring
- **Health checks**: Configurable intervals
- **Connection verification**: `monitor_ping_interval`
- **Replication lag detection**: Read/write splitting
- **Automatic shunning**: Removes unhealthy backends

## Performance and Scaling Architecture

### Connection Pooling Strategy
```mermaid
graph TD
    subgraph "Client Connections"
        C1[Client 1]
        C2[Client 2]
        C3[Client N]
    end
    
    subgraph "ProxySQL Connection Multiplexing"
        FE[Frontend Connections<br/>1000s possible]
        MUX[Multiplexer]
        BE[Backend Pool<br/>10s-100s connections]
    end
    
    subgraph "Backend Servers"
        HG0[HostGroup 0<br/>Writers]
        HG1[HostGroup 1<br/>Readers]
        HG2[HostGroup 2<br/>Analytics]
    end
    
    C1 --> FE
    C2 --> FE
    C3 --> FE
    FE --> MUX
    MUX --> BE
    BE --> HG0
    BE --> HG1
    BE --> HG2
    
    style FE fill:#e8f5e8
    style MUX fill:#fff3e0
    style BE fill:#e1f5fe
```

### Caching
- **Query Cache**: TTL and pattern-based
- **Prepared Statement Cache**: Metadata caching
- **Connection Attributes Cache**: Reduced authentication overhead

## Testing and Quality Assurance

### Test Infrastructure Layers
1. **Unit Tests**: Component-level testing
2. **TAP Tests**: 220+ Test Anything Protocol tests
3. **Integration Tests**: Full system scenarios
4. **Cluster Tests**: Multi-node configurations
5. **Fuzzing Tests**: AFL-based security testing

### Test Execution
```bash
# Run TAP tests
make build_tap_test_debug
cd test/tap/tests
./run_tests.sh

# Run specific test suite
./test_mysql_connect-t
./test_pgsql_authentication-t
```

## Build and Development Workflow

### Development Setup
```bash
# Clone and build with dependencies
git clone https://github.com/sysown/proxysql.git
cd proxysql
make build_deps
make debug

# Run with custom config
./src/proxysql -f -c proxysql.cnf

# Connect to admin interface
mysql -h127.0.0.1 -P6032 -uadmin -padmin
```

### Debug Build Features
- Full debugging symbols
- Assertion checks enabled
- Memory leak detection
- Core dump generation
- Verbose logging

## Architecture Features

1. **Protocol-Aware**: Understands and manipulates database protocols
2. **Runtime Reconfiguration**: Changes without connection drops
3. **Load Balancing**: Weight-based, least-connections, round-robin
4. **Query Routing**: Regex rules, rewriting, caching
5. **Integration**: Prometheus, REST API, clustering
6. **Security**: SSL/TLS, SQL injection detection, authentication
7. **Performance**: Lock-free statistics, connection pooling, caching
8. **High Availability**: Failover, read/write splitting, lag detection