# llms-full (private-aware)
> Built from GitHub files and website pages. Large files may be truncated.

--- docs-site/README.md ---
# ck Documentation Site

VitePress-powered documentation for the ck hybrid code search tool.

## Prerequisites

- Node.js 18+ or 20+
- pnpm 10+

## Installation

```bash
# Install pnpm if you haven't already
npm install -g pnpm

# Install dependencies
pnpm install
```

## Development

Start the local development server with hot reload:

```bash
pnpm dev
```

The site will be available at `http://localhost:5173` (or next available port).

## Building

Build the documentation site for production:

```bash
pnpm build
```

Output will be in `.vitepress/dist/`.

## Preview

Preview the production build locally:

```bash
pnpm build
pnpm preview
```

## Project Structure

```
docs-site/
├── .vitepress/
│   ├── config.ts           # Site configuration
│   ├── dist/               # Build output (gitignored)
│   └── cache/              # Build cache (gitignored)
├── guide/                  # Getting started guides
│   ├── introduction.md
│   ├── installation.md
│   ├── basic-usage.md
│   └── advanced-usage.md
├── features/               # Feature documentation
│   ├── semantic-search.md
│   ├── mcp-integration.md
│   ├── hybrid-search.md
│   └── grep-compatibility.md
├── reference/              # Reference documentation
│   ├── cli.md
│   ├── models.md
│   ├── configuration.md
│   └── architecture.md
├── contributing/           # Contributing guides
│   ├── development.md
│   ├── release-process.md
│   └── testing.md
├── index.md                # Landing page
├── package.json            # Dependencies and scripts
└── README.md               # This file
```

## Writing Documentation

### Markdown Features

VitePress supports enhanced markdown:

#### Code Blocks with Syntax Highlighting

\`\`\`rust
fn main() {
    println!(“Hello, world!”);
}
\`\`\`

#### Code Groups

\`\`\`rust
// Rust example
\`\`\`

\`\`\`python
# Python example
\`\`\`

#### Custom Containers

\`\`\`markdown
::: tip
This is a tip
:::

::: warning
This is a warning
:::

::: danger
This is a danger notice
:::
\`\`\`

#### Tables

```markdown
| Header 1 | Header 2 |
|----------|----------|
| Cell 1   | Cell 2   |
```

### Adding New Pages

1. Create markdown file in appropriate directory
2. Add to sidebar in `.vitepress/config.ts`:

```typescript
sidebar: {
  '/guide/': [
    {
      text: 'Guide',
      items: [
        { text: 'New Page', link: '/guide/new-page' }
      ]
    }
  ]
}
```

### Internal Links

```markdown
[Link text](/guide/installation)
[Another page](/features/semantic-search#section)
```

### Images

Place images in `public/` directory and reference:

```markdown
![Alt text](/image.png)
```

## Configuration

Main config file: `.vitepress/config.ts`

Key sections:
- `title`: Site title
- `description`: Site description
- `themeConfig.nav`: Top navigation
- `themeConfig.sidebar`: Sidebar navigation
- `themeConfig.search`: Search configuration

## Deployment

### GitHub Pages

Add to `.github/workflows/deploy-docs.yml`:

```yaml
name: Deploy Docs

on:
  push:
    branches: [main]
    paths:
      - 'docs-site/**'

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v2
        with:
          version: 10
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: 'pnpm'
          cache-dependency-path: docs-site/pnpm-lock.yaml
      - run: cd docs-site && pnpm install
      - run: cd docs-site && pnpm build
      - uses: peaceiris/actions-gh-pages@v3
        with:
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: docs-site/.vitepress/dist
```

### Netlify

1. Connect repository to Netlify
2. Set build settings:
   — Base directory: `docs-site`
   — Build command: `pnpm build`
   — Publish directory: `docs-site/.vitepress/dist`

### Vercel

```json
{
  "buildCommand": "cd docs-site && pnpm build",
  "outputDirectory": "docs-site/.vitepress/dist"
}
```

## Troubleshooting

### Port Already in Use

```bash
# Kill process on port 5173
lsof -ti:5173 | xargs kill -9

# Or use different port
pnpm dev --port 5174
```

### Build Errors

```bash
# Clear cache and rebuild
rm -rf .vitepress/cache .vitepress/dist node_modules
pnpm install
pnpm build
```

### Broken Links

VitePress checks for broken links during build. Fix any reported issues.

## Contributing to Docs

1. Fork the repository
2. Create feature branch
3. Make changes to docs
4. Test locally with `pnpm dev`
5. Build to verify: `pnpm build`
6. Submit pull request

## Resources

- [VitePress Documentation](https://vitepress.dev/)
- [Markdown Guide](https://vitepress.dev/guide/markdown)
- [Vue.js Documentation](https://vuejs.org/) (for custom components)

## License

MIT or Apache-2.0 (same as ck project)


## Links discovered
- [Link text](https://github.com/BeaconBay/ck/blob/main/guide/installation.md)
- [Another page](https://github.com/BeaconBay/ck/blob/main/features/semantic-search#section.md)
- [Alt text](https://github.com/BeaconBay/ck/blob/main/image.png)
- [VitePress Documentation](https://vitepress.dev/)
- [Markdown Guide](https://vitepress.dev/guide/markdown)
- [Vue.js Documentation](https://vuejs.org/)

--- docs-site/guide/installation.md ---
---
title: Installation
description: Install ck hybrid code search from NPM, crates.io, or source. Get started with your first search in minutes with automatic indexing and model downloads.
---

# Installation

Get up and running with ck in minutes.

## From NPM (Recommended)

```bash
npm install -g @beaconbay/ck-search
```

This installs the NPM package from [@beaconbay/ck-search](https://www.npmjs.com/package/@beaconbay/ck-search).

### Check for updates

```bash
# Check current version
npm list -g @beaconbay/ck-search

# Check if updates are available
npm outdated -g @beaconbay/ck-search

# Upgrade to the latest version
npm update -g @beaconbay/ck-search
```

## From crates.io

```bash
cargo install ck-search
```

This installs the latest stable release from [crates.io](https://crates.io/crates/ck-search).

## From Source

```bash
git clone https://github.com/BeaconBay/ck
cd ck
cargo install --path ck-cli
```

## Verify Installation

```bash
ck --version
```

## Your First Search

ck works just like grep — no configuration needed:

```bash
# Traditional keyword search
ck "TODO" src/

# Semantic search (automatically builds index on first run)
ck --sem "error handling" src/
```

::: tip First-Time Setup
The first semantic search will:
1. Detect your project structure
2. Download embedding model (one-time, ~80MB)
3. Index your codebase
4. Perform the search

Subsequent searches are fast — only changed files are re-indexed.
:::

## Quick Examples

### Semantic Search

Find code by concept:

```bash
# Find error handling patterns
ck --sem "error handling" src/

# Find authentication code
ck --sem "user authentication" src/

# Find database queries
ck --sem "SQL queries" src/

# Get complete functions
ck --sem --full-section "retry logic" src/
```

### grep-Compatible Search

All standard grep flags work:

```bash
# Case-insensitive search
ck -i "warning" src/

# Show line numbers and context
ck -n -A 3 -B 1 "error" src/

# List files with matches
ck -l "TODO" src/

# Recursive with pattern
ck -R "bug|fix" .
```

### Hybrid Search

Combine semantic and keyword search:

```bash
# Best of both worlds
ck --hybrid "connection timeout" src/

# Show relevance scores
ck --hybrid --scores "cache invalidation" src/

# Filter by confidence
ck --hybrid --threshold 0.02 "auth" src/
```

*Note: Hybrid search uses RRF (Reciprocal Rank Fusion) scores in the 0.01-0.05 range, unlike semantic search which uses 0.0-1.0.*

## Understanding the Output

### Standard Output

```bash
$ ck "error" src/main.rs
src/main.rs:42:    let result = risky_operation().map_err(|e| {
src/main.rs:43:        eprintln!("Error: {}", e);
```

### With Semantic Scores

```bash
$ ck --sem --scores "error handling" src/
[0.847] ./error_handler.rs: Comprehensive error handling with custom types
[0.732] ./main.rs: Main application with error propagation
[0.651] ./utils.rs: Utility functions with Result returns
```

Higher scores indicate stronger semantic similarity (0.0 — 1.0).

## Common Workflows

### Finding Related Code

```bash
# Find all authentication-related code
ck --sem "authentication" .

# Find test files for a feature
ck --sem "unit tests for auth" tests/

# Find configuration handling
ck --sem "config parsing" src/
```

### Code Review

```bash
# Find potential security issues
ck --hybrid "sql injection|xss" src/

# Find missing error handling
ck -L "Result|Option" src/*.rs

# Find TODOs from recent changes
git diff --name-only | xargs ck "TODO"
```

### Exploring Unfamiliar Codebases

```bash
# Understand project structure
ck --sem "main entry point" .

# Find similar functionality
ck --sem --full-section "http request handler" src/

# Locate business logic
ck --sem "payment processing" src/
```

## File Exclusions

ck automatically excludes:
- Binary files and build artifacts
- `.git` directories
- Files in `.gitignore`
- Media files (images, videos, audio)
- Common cache directories

### Custom Exclusions

```bash
# Exclude specific patterns
ck --exclude "*.test.js" --sem "api" src/

# Disable gitignore
ck --no-ignore "pattern" .

# Edit .ckignore file
vim .ckignore  # Uses gitignore syntax
```

## Index Management

ck automatically manages indexes, but you can control them:

```bash
# Check index status
ck --status .

# Force rebuild
ck --clean .

# Add single file
ck --add new_file.rs

# Inspect chunking strategy
ck --inspect src/main.rs
```

::: warning
`ck --clean` removes the entire index and requires a full rebuild. For most cases, use `ck --index` instead — it updates incrementally and is much faster.
:::

## Model Selection

Choose embedding models for different needs:

```bash
# Default: BGE-Small (fast, precise)
ck --index .

# Large contexts: Nomic V1.5
ck --index --model nomic-v1.5 .

# Code-specialized: Jina Code
ck --index --model jina-code .
```

See [Embedding Models](/reference/models) for detailed comparison.

## Choosing Your Interface

ck offers multiple interfaces for different workflows:

### Command-Line Interface (CLI)

The default mode you’ve been using — perfect for scripts and pipelines:

```bash
ck --sem "pattern" src/
```

### Terminal User Interface (TUI)

Interactive exploration with live results:

```bash
# Launch TUI mode
ck-tui

# Then:
# - Type queries and see live results
# - Navigate with ↑/↓
# - Preview with →
# - Open files with Enter
```

The TUI provides:
- Live search as you type
- Visual result previews
- Keyboard-driven navigation
- Score heatmaps
- Multiple preview modes

See [TUI Mode](/features/tui-mode) for complete documentation.

### Editor Integration (VSCode/Cursor)

Search without leaving your editor:

```bash
# Install extension (see docs for details)
code --install-extension ck-search

# Then use:
# - Cmd+Shift+; to open search
# - Cmd+Shift+' to search selection
```

See [Editor Integration](/features/editor-integration) for setup and usage.

### MCP Server (AI Agents)

Integrate with Claude Desktop and other AI tools:

```bash
# Start MCP server
ck --serve

# Then configure in Claude Desktop
```

See [MCP Integration](/features/mcp-integration) for complete setup, or [AI Agent Setup](/guide/ai-agent-setup) for configuration best practices with Claude Code and other AI coding assistants.

### Which Interface?

- **CLI**: Scripts, automation, grep replacement
- **TUI**: Interactive exploration, code discovery
- **Editor**: In-editor search, zero context switch
- **MCP**: AI-assisted code understanding

Read the full [Choosing an Interface](/guide/choosing-interface) guide for detailed comparison.

## Next Steps

- Learn [basic usage patterns](/guide/basic-usage)
- Try [TUI mode](/features/tui-mode) for interactive exploration
- Install [editor extension](/features/editor-integration) for in-editor search
- Configure [AI agent setup](/guide/ai-agent-setup) for Claude Code and AI assistants
- Explore [advanced features](/guide/advanced-usage)
- Set up [MCP integration](/features/mcp-integration)
- Check the [CLI reference](/reference/cli)

## Troubleshooting

### First Index Takes Long

First-time indexing downloads models and processes all files. Subsequent searches only process changed files.

### Model Download Fails

::: tip Model Cache Location
Models are cached in:
- Linux/macOS: `~/.cache/ck/models/`
- Windows: `%LOCALAPPDATA%\ck\cache\models\`

Ensure you have an active internet connection and ~500MB free disk space.
:::

### Search Results Seem Wrong

Try different search modes:
```bash
# Try hybrid instead of pure semantic
ck --hybrid "your query" .

# Adjust threshold
ck --sem --threshold 0.3 "query" .

# Use keyword search
ck "exact phrase" .
```

See [Configuration](/reference/configuration) for tuning options.


## Links discovered
- [@beaconbay/ck-search](https://www.npmjs.com/package/@beaconbay/ck-search)
- [crates.io](https://crates.io/crates/ck-search)
- [Embedding Models](https://github.com/BeaconBay/ck/blob/main/reference/models.md)
- [TUI Mode](https://github.com/BeaconBay/ck/blob/main/features/tui-mode.md)
- [Editor Integration](https://github.com/BeaconBay/ck/blob/main/features/editor-integration.md)
- [MCP Integration](https://github.com/BeaconBay/ck/blob/main/features/mcp-integration.md)
- [AI Agent Setup](https://github.com/BeaconBay/ck/blob/main/guide/ai-agent-setup.md)
- [Choosing an Interface](https://github.com/BeaconBay/ck/blob/main/guide/choosing-interface.md)
- [basic usage patterns](https://github.com/BeaconBay/ck/blob/main/guide/basic-usage.md)
- [TUI mode](https://github.com/BeaconBay/ck/blob/main/features/tui-mode.md)
- [editor extension](https://github.com/BeaconBay/ck/blob/main/features/editor-integration.md)
- [AI agent setup](https://github.com/BeaconBay/ck/blob/main/guide/ai-agent-setup.md)
- [advanced features](https://github.com/BeaconBay/ck/blob/main/guide/advanced-usage.md)
- [MCP integration](https://github.com/BeaconBay/ck/blob/main/features/mcp-integration.md)
- [CLI reference](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)
- [Configuration](https://github.com/BeaconBay/ck/blob/main/reference/configuration.md)

--- docs-site/index.md ---
---
layout: home

hero:
  name: ck <span style="font-weight:300;">("seek")</span>
  text: Hybrid Code Search
  tagline: ck’s hybrid search fuses lexical (BM25/grep) precision with embedding-based recall and re-ranks both, so you find the right code even when the exact keywords aren’t there.
  image:
    src: /logo.png
    alt: ck logo
  actions:
    - theme: brand
      text: Get Started
      link: /guide/installation
    - theme: alt
      text: View on GitHub
      link: https://github.com/BeaconBay/ck

features:
  - icon: ⚡
    title: Drop-in <code>grep</code> Replacement
    details: All your muscle memory works. Same flags, same behavior, same output format — plus semantic understanding when you need it
    link: /features/grep-compatibility

  - icon: 🔍
    title: Semantic Search
    details: Find code by concept, not keywords. Search for “retry logic” and find backoff, circuit breakers, and related patterns even without exact matches
    link: /features/semantic-search

  - icon: 🎯
    title: Hybrid Search
    details: Combine keyword precision with semantic understanding using Reciprocal Rank Fusion for best-of-both-worlds search results
    link: /features/hybrid-search

  - icon: 🤖
    title: AI Agent Integration
    details: Built-in MCP (Model Context Protocol) server for seamless integration with Claude Desktop, Cursor, and any MCP-compatible AI client
    link: /features/mcp-integration

  - icon: 💻
    title: Terminal User Interface
    details: Interactive search with live results, visual score heatmaps, and keyboard-driven navigation. Explore code with TUI mode for instant feedback
    link: /features/tui-mode

  - icon: 🔌
    title: Editor Integration
    details: Native VSCode and Cursor extension. Search without leaving your editor with inline results, instant navigation, and live updates
    link: /features/editor-integration

  - icon: 🚀
    title: Blazing Fast
    details: ~1M LOC indexed in under 2 minutes. Sub-500ms queries. Chunk-level incremental indexing only re-embeds what changed
    link: /guide/basic-usage

  - icon: 📦
    title: Completely Offline
    details: Everything runs locally. No code or queries sent to external services. Embedding model downloaded once and cached locally
    link: /reference/models
---

## Quick Start

```bash
# Install from NPM
npm install -g @beaconbay/ck-search

# CLI: Command-line search (grep-compatible)
ck --sem "error handling" src/
ck --hybrid "connection timeout" src/
ck -n "TODO" *.rs

# TUI: Interactive terminal UI
ck-tui
# Type queries, see live results, navigate with ↑/↓

# Editor: VSCode/Cursor extension
code --install-extension ck-search
# Press Cmd+Shift+; to search

# MCP: AI agent integration
ck --serve
# Configure in Claude Desktop for AI-assisted search
```

## Why ck?

**ck (seek)** finds code by meaning, not just keywords. It’s the grep you wish you had:

- 🎯 **Understand intent** – Search for “error handling” and find try/catch blocks, error returns, and exception handling even when those exact words aren’t present
- 🤖 **AI-first** – Built-in MCP server for direct integration with AI coding assistants
- ⚡ **Fast & efficient** – Automatic incremental indexing, sub-second queries
- 🔧 **Drop-in replacement** – Works exactly like grep/ripgrep with all the flags you know
- 🌐 **Multi-language** – Python, JavaScript/TypeScript, Rust, Go, Ruby, Haskell, C#, and more
- 🔒 **Privacy-first** – 100% offline, no telemetry, no external API calls

## Installation

### From NPM (recommended)
```bash
npm install -g @beaconbay/ck-search
```

### From crates.io
```bash
cargo install ck-search
```

### From source
```bash
git clone https://github.com/BeaconBay/ck
cd ck
cargo install --path ck-cli
```

## Next Steps

<div class="vp-doc">

- [**Getting Started Guide**](/guide/installation) — Installation and first search
- [**Choosing an Interface**](/guide/choosing-interface) — CLI, TUI, Editor, or MCP?
- [**TUI Mode**](/features/tui-mode) — Interactive terminal interface
- [**Editor Integration**](/features/editor-integration) — VSCode/Cursor extension
- [**MCP Integration**](/features/mcp-integration) — Connect with AI agents
- [**Basic Usage**](/guide/basic-usage) — Common patterns and workflows
- [**CLI Reference**](/reference/cli) — Complete command-line reference

</div>


## Links discovered
- [**Getting Started Guide**](https://github.com/BeaconBay/ck/blob/main/guide/installation.md)
- [**Choosing an Interface**](https://github.com/BeaconBay/ck/blob/main/guide/choosing-interface.md)
- [**TUI Mode**](https://github.com/BeaconBay/ck/blob/main/features/tui-mode.md)
- [**Editor Integration**](https://github.com/BeaconBay/ck/blob/main/features/editor-integration.md)
- [**MCP Integration**](https://github.com/BeaconBay/ck/blob/main/features/mcp-integration.md)
- [**Basic Usage**](https://github.com/BeaconBay/ck/blob/main/guide/basic-usage.md)
- [**CLI Reference**](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)

--- ck-vscode/GETTING_STARTED.md ---
# Getting Started with ck VS Code Extension

## Quick Start

### 1. Install Dependencies

```bash
cd ck-vscode
npm install
```

### 2. Compile the Extension

```bash
npm run compile
```

This compiles TypeScript to JavaScript in the `out/` directory.

### 3. Test the Extension

1. Open the `ck-vscode` folder in VS Code
2. Press `F5` to launch the Extension Development Host
3. In the new window, open a workspace folder
4. Click the ck icon in the activity bar (left sidebar)
5. Start searching!

### 4. Make Changes

- Edit files in `src/` - the extension TypeScript code
- Edit files in `webview/` - the search panel UI
- Run `npm run watch` to auto-compile on changes
- Reload the Extension Development Host with `Ctrl+R` / `Cmd+R`

## Architecture Overview

```
ck-vscode/
├── src/
│   ├── extension.ts       # Extension entry point, registers commands
│   ├── searchPanel.ts     # Webview provider, manages search UI
│   ├── cliAdapter.ts      # Spawns ck binary, parses results
│   └── types.ts           # TypeScript interfaces
├── webview/
│   ├── main.js           # Webview UI logic
│   ├── styles.css        # TUI-inspired styling
│   └── (index.html is generated by searchPanel.ts)
├── package.json          # Extension manifest, commands, config
└── tsconfig.json         # TypeScript configuration
```

## How It Works

### CLI Mode (Phase 1 - Current)

1. User types in search box
2. After 300ms debounce, webview sends message to extension
3. Extension spawns `ck --jsonl --hybrid "query" /path`
4. ck outputs JSONL results to stdout
5. Extension parses results and sends to webview
6. Webview renders results
7. User clicks result → extension opens file at line

### Message Flow

```
Webview                Extension              ck Binary
   |                       |                      |
   |--{ type: 'search' }-->|                      |
   |                       |--spawn("ck ...")---->|
   |                       |                      |
   |                       |<--JSONL results------|
   |<--{ searchResults }---|                      |
   |                       |                      |
   |--{ type: 'openFile'}->|                      |
   |                       |--vscode.open()       |
```

## Testing Checklist

- [ ] Search with hybrid mode works
- [ ] Search with semantic mode works
- [ ] Search with regex mode works
- [ ] Clicking result opens file at correct line
- [ ] Keyboard navigation (↑/↓) works
- [ ] Enter key opens selected result
- [ ] Debouncing works (no spam)
- [ ] Reindex button works
- [ ] Index status updates correctly
- [ ] Search selection command works
- [ ] Extension activates on startup

## Next Steps (Phase 2+)

### MCP Integration

To add MCP support:

1. Create `src/mcpAdapter.ts` - JSON-RPC client for `ck --serve`
2. Update `searchPanel.ts` to use MCP or CLI based on config
3. Add streaming support for incremental results
4. Add progress notifications

### Enhancements

- Syntax highlighting in result previews (use VS Code's TextMate grammars)
- Peek view for inline results
- Multi-workspace support
- Result filtering UI
- Search history

## Packaging for Distribution

```bash
# Install vsce if not already installed
npm install -g @vscode/vsce

# Package the extension
npm run package

# This creates ck-search-0.1.0.vsix
```

Install locally:
```bash
code --install-extension ck-search-0.1.0.vsix
```

## Troubleshooting

### Extension doesn't activate
- Check the Output panel (View → Output) and select "ck"
- Look for errors in Developer Tools (Help → Toggle Developer Tools)

### ck binary not found
- Ensure `ck` is in PATH: `which ck` or `where ck`
- Set `ck.cliPath` in settings to absolute path

### Search returns no results
- Check ck works in terminal: `ck --sem "test" .`
- Look at stderr output in Developer Tools console

### Webview not loading
- Check CSP errors in Developer Tools
- Verify webview resources exist in `webview/` directory

## Development Tips

- Use `console.log()` in webview code - it appears in Developer Tools
- Use `console.log()` in extension code - it appears in Debug Console
- Set breakpoints in TypeScript files
- Use `vscode.window.showInformationMessage()` for quick debugging
- Check the Extension Host log: View → Output → Extension Host

Enjoy building! 🚀


--- docs-site/reference/advanced.md ---
---
title: Advanced Configuration
description: Advanced ck configuration including environment variables, index optimization, shell integration, and power-user techniques for semantic search.
---

# Advanced Configuration

Advanced configuration options, environment variables, and power-user techniques for ck.

## Environment Variables

### HuggingFace Cache Reuse

Reuse your existing HuggingFace cache instead of downloading models again:

```bash
# Set HuggingFace home directory
export HF_HOME=~/.cache/huggingface

# Or set hub cache directly
export HF_HUB_CACHE=~/.cache/huggingface/hub

# Now ck will use existing cached models
ck --index .
```

**Use case**: You already have embedding models downloaded for other projects.

**Benefits**:
- Avoid duplicate downloads
- Save disk space
- Faster initial setup

### Model Cache Locations

ck/fastembed uses these cache directories:

| Platform | Default Location |
|----------|------------------|
| Linux | `~/.cache/ck/models/` |
| macOS | `~/.cache/ck/models/` |
| Windows | `%LOCALAPPDATA%\ck\cache\models\` |
| Fallback | `.ck_models/models/` (current directory) |

**Override with environment variables**:
```bash
export HF_HOME=/custom/cache/dir
```

### Rust Logging

Enable detailed logging for debugging:

```bash
# All logs
RUST_LOG=debug ck --sem "pattern" src/

# Specific module
RUST_LOG=ck_engine=trace ck --sem "pattern" src/

# Multiple modules
RUST_LOG=ck_engine=debug,ck_index=info ck --index .
```

**Log levels**: `error`, `warn`, `info`, `debug`, `trace`

## Performance Tuning

### Large Codebase Strategies

For codebases >1M LOC:

#### 1. Choose Faster Model

```bash
# Use BGE-Small (fastest)
ck --index --model bge-small .
```

BGE-Small trades some accuracy for speed:
- 400-token chunks (vs 1024 for others)
- Smaller embedding dimensions (384 vs 768)
- Faster inference

#### 2. Aggressive Exclusions

Create comprehensive `.ckignore`:

```txt
# Build artifacts
target/
dist/
build/
out/

# Dependencies
node_modules/
vendor/
venv/
.venv/

# Generated code
*.generated.*
*_pb2.py
*.pb.go

# Documentation
docs/
*.md
README*

# Tests (if you don't search them)
**/test/
**/tests/
**/*_test.*
**/*.test.*

# Data files
*.json
*.yaml
*.csv
*.xml

# Media
*.png
*.jpg
*.gif
*.svg
*.mp4
```

#### 3. Index in Batches

For extremely large monorepos:

```bash
# Index core first
echo "/*" > .ckignore
echo "!/src/" >> .ckignore
ck --index .

# Add more directories incrementally
echo "!/lib/" >> .ckignore
ck --index .

# And so on...
```

#### 4. Separate Indexes for Components

For microservices/monorepos, index each service separately:

```bash
cd service-a && ck --index .
cd ../service-b && ck --index .
# etc.
```

### Memory Optimization

#### Reduce Chunk Size

For memory-constrained environments:

```bash
# Use smaller model with smaller chunks
ck --index --model bge-small .
```

#### Limit Search Results

```bash
# Limit to top 10 results
ck --sem --topk 10 "pattern" src/

# Use aggressive threshold
ck --sem --threshold 0.8 "pattern" src/
```

#### JSONL Without Snippets

For AI agents with tight memory budgets:

```bash
# Metadata only, no content
ck --jsonl --no-snippet --sem "pattern" src/
```

### Disk Space Optimization

#### Clean Unused Indexes

```bash
# Find all .ck directories
find . -type d -name ".ck" -exec du -sh {} \;

# Remove indexes from old branches
find . -type d -name ".ck" -path "*/old-feature/*" -exec rm -rf {} \;
```

#### Use Symlinks for Shared Models

If working with multiple repositories:

```bash
# Central model cache
mkdir -p ~/shared-ck-models

# Symlink from each repo
cd ~/project1
ln -s ~/shared-ck-models ~/.cache/ck/models

cd ~/project2
ln -s ~/shared-ck-models ~/.cache/ck/models
```

## Intelligent Code Chunking

ck uses tree-sitter for language-aware code parsing and chunking, ensuring semantic boundaries are respected.

### How Chunking Works

**Tree-sitter Query-Based Chunking**:

ck parses code using tree-sitter grammars and extracts meaningful units (functions, classes, methods) as chunks:

```bash
# View how a file is chunked
ck --inspect src/auth.rs
```

**Example output**:
```
File: src/auth.rs
Language: Rust
Chunks: 12

Chunk 1 (lines 1-15, 342 tokens)
  Type: function
  Content: fn authenticate_user(...)

Chunk 2 (lines 17-45, 528 tokens)
  Type: struct + impl
  Content: struct AuthService { ... }

Chunk 3 (lines 47-62, 287 tokens)
  Type: function
  Content: fn validate_token(...)
```

### Supported Languages

Tree-sitter chunking available for:

| Language | Chunks | Grammar |
|----------|--------|---------|
| Python | Functions, classes, methods | `tree-sitter-python` |
| JavaScript/TypeScript | Functions, classes, methods, arrow functions | `tree-sitter-javascript`, `tree-sitter-typescript` |
| Rust | Functions, structs, impls, traits | `tree-sitter-rust` |
| Go | Functions, methods, structs | `tree-sitter-go` |
| Ruby | Methods, classes, modules | `tree-sitter-ruby` |
| Haskell | Functions, type declarations | `tree-sitter-haskell` |
| C# | Methods, classes, namespaces | `tree-sitter-c-sharp` |
| Zig | Functions, structs | `tree-sitter-zig` |

Unsupported languages fall back to size-based chunking.

### Chunk-Level Incremental Indexing

**Introduced in v0.7.0**, ck performs incremental indexing at the chunk level:

**How it works**:
1. Each chunk gets a hash (blake3 of text + trivia)
2. When files change, only modified chunks are re-embedded
3. Unchanged chunks reuse cached embeddings
4. Dramatically faster re-indexing

**Benefits**:
```bash
# Initial index: 100 files, 2000 chunks → 60 seconds
ck --index .

# Edit 1 function in 1 file → only 1 chunk re-embedded
# Edit file → ~2 seconds (vs 60 for full rebuild)
ck --index .
```

**Cache invalidation**:
- Chunk text changes → re-embed
- Doc comments change → re-embed
- Whitespace within function → re-embed
- Other chunks in same file → cached (not re-embedded)

### Chunk Configuration by Model

Different embedding models use different chunk sizes:

| Model | Chunk Size | Context Window | Use Case |
|-------|-----------|----------------|----------|
| BGE-Small | 400 tokens | 512 | Fast, precise functions |
| Nomic V1.5 | 1024 tokens | 8192 | Large context, whole classes |
| Jina Code | 1024 tokens | 8192 | Code-specialized, documentation |

**Model selection affects chunking**:

```bash
# Small chunks (400 tokens) - more granular
ck --index --model bge-small .

# Large chunks (1024 tokens) - more context
ck --index --model nomic-v1.5 .
```

### Trivia Handling

**Leading and trailing trivia** (comments, whitespace) are included in chunks:

**Python example**:
```python
# This is a doc comment  ← Trivia (included in chunk hash)
def authenticate_user(token):  ← Code
    """Validate user token"""  ← Trivia
    return validate(token)
```

**Why trivia matters**:
- Doc comments change semantic meaning
- Preserves context for AI understanding
- Ensures cache invalidation on meaningful changes

### Chunk Inspection

Use `--inspect` to understand chunking:

```bash
# Single file
ck --inspect src/main.rs

# Compare chunking across models
ck --inspect src/auth.py --model bge-small
ck --inspect src/auth.py --model nomic-v1.5
```

**What to look for**:
- **Chunk count**: Too many chunks = consider larger model
- **Token counts**: Near limit = may be truncated
- **Chunk boundaries**: Should align with functions/classes
- **Language detection**: Verify correct parser used

### Handling Edge Cases

**Large functions** (>1024 tokens):
- Automatically split with overlap (striding)
- Both chunks reference same location
- Ensures no content loss

**Mixed-language files** (e.g., HTML with JavaScript):
- Uses extension-based language detection
- Falls back to text chunking if ambiguous
- Can be tuned with explicit patterns

**Generated code**:
- Often produces many similar chunks
- Consider excluding in `.ckignore`:
  ```txt
  *.generated.*
  *_pb2.py
  *.pb.go
  ```

## Advanced Search Patterns

### Custom Threshold Strategies

Different thresholds for different use cases:

```bash
# Exploratory (cast wide net)
ck --sem --threshold 0.3 --topk 50 "vague concept" src/

# Precise (only high confidence)
ck --sem --threshold 0.8 --topk 5 "specific pattern" src/

# Balanced (default)
ck --sem --threshold 0.6 --topk 20 "pattern" src/
```

### Combining Search Modes

```bash
# Semantic search then keyword filter
ck --sem "authentication" src/ | grep -i "jwt"

# Hybrid with custom output
ck --hybrid --scores "pattern" src/ | awk '$1 > 0.7'

# Multiple searches combined
{
  ck --sem "auth" src/
  ck --sem "login" src/
} | sort -u
```

### Pipeline Integration

#### Git Hooks

**Pre-commit hook** (`.git/hooks/pre-commit`):
```bash
#!/bin/bash

# Find TODOs in staged files
TODOS=$(git diff --cached --name-only | xargs ck --sem "TODO" 2>/dev/null)

if [ -n "$TODOS" ]; then
    echo "Warning: Found TODOs in staged files:"
    echo "$TODOS"
    read -p "Commit anyway? (y/n) " -n 1 -r
    echo
    if [[ ! $REPLY =~ ^[Yy]$ ]]; then
        exit 1
    fi
fi
```

**Pre-push hook** (`.git/hooks/pre-push`):
```bash
#!/bin/bash

# Check for potential security issues
SECURITY=$(ck --hybrid "password|api_key|secret" $(git diff --name-only origin/main...HEAD))

if [ -n "$SECURITY" ]; then
    echo "Error: Potential security issues found:"
    echo "$SECURITY"
    exit 1
fi
```

#### CI/CD Integration

**GitHub Actions**:
```yaml
- name: Install ck
  run: cargo install ck-search

- name: Check for security issues
  run: |
    if ck --hybrid "password|secret|api_key" src/; then
      echo "::error::Found potential security issues"
      exit 1
    fi

- name: Verify test coverage
  run: |
    # Find source files without corresponding tests
    if ck -L --sem "test" src/**/*.rs; then
      echo "::warning::Files without tests found"
    fi
```

**GitLab CI**:
```yaml
semantic_audit:
  script:
    - cargo install ck-search
    - ck --sem "TODO|FIXME" src/ || true
    - ck --hybrid "deprecated" src/ || true
  artifacts:
    reports:
      - audit_report.txt
```

### Shell Integration

#### Aliases

Add to `.bashrc` or `.zshrc`:

```bash
# Semantic search aliases
alias cks='ck --sem'
alias ckh='ck --hybrid'
alias ckf='ck --sem --full-section'

# Common patterns
alias ck-todo='ck --hybrid "TODO|FIXME|XXX"'
alias ck-bug='ck --sem "potential bug"'
alias ck-auth='ck --sem "authentication"'

# Search with scores
alias ckscore='ck --sem --scores --threshold 0.7'

# High-confidence search
alias ckstrict='ck --sem --threshold 0.8 --topk 5'
```

#### Functions

```bash
# Search and open in editor
ckopen() {
    local query="$1"
    local file=$(ck --sem "$query" . | fzf | cut -d: -f1)
    [ -n "$file" ] && $EDITOR "$file"
}

# Compare search modes
ckcompare() {
    local query="$1"
    echo "=== Semantic ==="
    ck --sem "$query" src/
    echo "\n=== Hybrid ==="
    ck --hybrid "$query" src/
    echo "\n=== Keyword ==="
    ck "$query" src/
}

# Index all git repos in directory
ck-index-all() {
    find . -name ".git" -type d | while read gitdir; do
        repo=$(dirname "$gitdir")
        echo "Indexing $repo"
        (cd "$repo" && ck --index .)
    done
}
```

## Advanced .ckignore Patterns

### Monorepo Patterns

```txt
# Exclude everything by default
/*

# Include only specific services
!/services/api/
!/services/web/
!/shared/

# But exclude their tests and mocks
services/**/test/
services/**/mocks/
```

### Framework-Specific Patterns

**Next.js**:
```txt
.next/
out/
*.config.js
public/
**/_app.tsx
**/_document.tsx
```

**Rails**:
```txt
db/schema.rb
db/migrate/
spec/fixtures/
tmp/
log/
```

**Django**:
```txt
**/migrations/
manage.py
**/settings.py
```

### Development vs Production

**Development** (`.ckignore.dev`):
```txt
# Aggressive exclusions for fast iteration
*.test.*
*.spec.*
**/__tests__/
docs/
*.md
```

**Production** (`.ckignore.prod`):
```txt
# Minimal exclusions for comprehensive search
node_modules/
dist/
*.log
```

Use with:
```bash
# Development
cp .ckignore.dev .ckignore && ck --index .

# Production
cp .ckignore.prod .ckignore && ck --index .
```

## Edge Cases & Workarounds

### Very Large Files

For files >10K LOC that slow indexing:

**Option 1**: Exclude from index
```txt
# .ckignore
schema.sql
generated/
```

**Option 2**: Split if possible
```bash
# Split large file into chunks
split -l 5000 large_file.py chunk_

# Index separately
ck --add chunk_*
```

**Option 3**: Use keyword search
```bash
# Skip semantic indexing for specific patterns
ck "specific_function" large_file.sql
```

### Mixed Language Repositories

For repos with many languages:

```bash
# Index supported languages first
echo "*.java" >> .ckignore
echo "*.php" >> .ckignore
ck --index .

# Use keyword search for unsupported
ck "pattern" **/*.java
```

### Symlinked Directories

ck follows symlinks by default. To exclude:

```bash
# Find symlinks
find . -type l

# Add to .ckignore
echo "symlinked-dir/" >> .ckignore
```

### Binary Files Incorrectly Detected as Text

Rare edge case where binary is indexed:

```txt
# .ckignore - explicit binary exclusions
*.bin
*.dat
*.exe
*.dll
*.so
*.dylib
specific-binary-file
```

## Debugging & Troubleshooting

### Verbose Indexing

```bash
# See what's being indexed
RUST_LOG=ck_index=debug ck --index . 2>&1 | grep "Processing"

# See chunking details
ck --inspect src/file.rs
```

### Check Index Health

```bash
# Status
ck --status .

# Verify file count
find . -type f -name "*.rs" | wc -l
# Compare with index status

# Check index size
du -sh .ck/
```

### Force Rebuild

When index seems corrupted:

```bash
# Nuclear option
rm -rf .ck/
ck --index .

# Or use clean command
ck --clean .
ck --index .
```

### Compare Models

```bash
# Script to compare models
for model in bge-small nomic-v1.5 jina-code; do
    echo "=== $model ==="
    ck --switch-model $model --force .
    time ck --sem "authentication" src/ | wc -l
done
```

## Security Considerations

### Sensitive Data

Never commit indexes of sensitive codebases:

```txt
# .gitignore
.ck/
```

### Access Control

For shared development machines:

```bash
# Restrict index directory permissions
chmod 700 .ck/

# Or use per-user cache
export HF_HOME=~/private/.ck-cache
```

### API Key Scanning

Regular semantic scans for secrets:

```bash
# Scheduled check
crontab -e
0 2 * * * cd /path/to/repo && ck --hybrid "api_key|password|secret" . | mail -s "Security Scan" dev@company.com
```

## Next Steps

- [Configuration](/reference/configuration) — Standard configuration options
- [CLI Reference](/reference/cli) — Complete command reference
- [Models](/reference/models) — Embedding model details
- [FAQ](/guide/faq) — Common questions and answers


## Links discovered
- [Configuration](https://github.com/BeaconBay/ck/blob/main/reference/configuration.md)
- [CLI Reference](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)
- [Models](https://github.com/BeaconBay/ck/blob/main/reference/models.md)
- [FAQ](https://github.com/BeaconBay/ck/blob/main/guide/faq.md)

--- docs-site/guide/advanced-usage.md ---
---
title: Advanced Usage
description: Power-user features for ck including model selection, complete code sections, custom exclusions, JSON/JSONL output, and CI/CD integration patterns.
---

# Advanced Usage

Power-user features and advanced workflows for ck.

## Model Selection

Choose the right embedding model for your needs.

### Available Models

```bash
# BGE-Small (default) - Fast, precise, 400-token chunks
ck --index --model bge-small .

# Nomic V1.5 - Large contexts, 1024-token chunks, 8K capacity
ck --index --model nomic-v1.5 .

# Jina Code - Code-specialized, 1024-token chunks, 8K capacity
ck --index --model jina-code .
```

### Model Comparison

| Model | Chunk Size | Context Window | Best For |
|-------|------------|----------------|----------|
| `bge-small` | 400 tokens | 512 tokens | General code, fast indexing |
| `nomic-v1.5` | 1024 tokens | 8K tokens | Large functions, documentation |
| `jina-code` | 1024 tokens | 8K tokens | Code-specific understanding |

### Switching Models

```bash
# Switch to different model
ck --switch-model nomic-v1.5 .

# Force rebuild (if unsure about index state)
ck --switch-model jina-code --force .

# Check current model
ck --status .
```

## Complete Code Sections

Extract entire functions, classes, or modules:

```bash
# Get full functions containing matches
ck --sem --full-section "database query" src/

# Works with regex too
ck --full-section "class.*Handler" src/

# Combine with other flags
ck --sem --full-section --scores "authentication" src/
```

This uses tree-sitter parsing to return complete syntactic units.

## Advanced Filtering

### Relevance Thresholds

```bash
# Only high-confidence matches (0.7+)
ck --sem --threshold 0.7 "auth" src/

# Exploratory search (0.3+)
ck --sem --threshold 0.3 "pattern" src/

# Very strict (0.9+)
ck --sem --threshold 0.9 "exact concept" src/
```

Scores range from 0.0 to 1.0, with higher being more relevant.

### Result Limiting

```bash
# Top 5 results
ck --sem --topk 5 "pattern" src/

# Alternative flag name
ck --sem --limit 10 "pattern" src/

# Combine with threshold
ck --sem --topk 20 --threshold 0.5 "pattern" src/
```

### Pagination for MCP/Agents

```bash
# First page (25 results)
ck --sem --page-size 25 "pattern" src/

# Get specific page
ck --sem --page-size 25 --cursor "abc123" "pattern" src/
```

Used primarily by MCP server for large result sets.

## Chunking Inspection

Understand how ck processes your files:

```bash
# Inspect file chunking
ck --inspect src/main.rs

# See token counts per chunk
ck --inspect src/large_file.py

# Test different models
ck --inspect --model nomic-v1.5 src/main.rs
```

Output shows:
- Detected language
- Number of chunks
- Token count per chunk
- Chunk boundaries

## Custom Exclusions

### .ckignore Syntax

Uses gitignore-style patterns:

```txt
# Exclude directory and all contents
node_modules/
target/
dist/

# Exclude file patterns
*.log
*.tmp
*.bak

# Exclude specific files
config/secrets.yaml
.env*

# Negation (include despite parent exclusion)
!important.log
```

### Multiple Exclusion Layers

ck combines exclusions from multiple sources:

```bash
# Default exclusions + .gitignore + .ckignore + CLI
ck --exclude "temp/" "pattern" src/

# Skip .gitignore
ck --no-ignore --exclude "temp/" "pattern" src/

# Skip .ckignore
ck --no-ckignore --exclude "temp/" "pattern" src/

# Skip both (only CLI + defaults)
ck --no-ignore --no-ckignore --exclude "temp/" "pattern" src/
```

All exclusion layers are additive.

## Index Management

### Incremental Updates

ck automatically updates indexes incrementally:

```bash
# First search: full index
ck --sem "pattern" src/

# Subsequent searches: only changed files
ck --sem "another pattern" src/
```

Uses file hashing to detect changes.

### Manual Index Control

```bash
# Force full rebuild
ck --clean .
ck --index .

# Add single file to existing index
ck --add src/new_file.rs

# Check what needs updating
ck --status .
```

### Index Location

Indexes stored in `.ck/` directories:

```bash
project/
├── src/
├── .ck/              # Safe to delete anytime
│   ├── embeddings.json
│   ├── ann_index.bin
│   └── tantivy_index/
└── .ckignore
```

The `.ck/` directory is a cache and can be safely deleted.

## Structured Output for Automation

### JSON Output

Single JSON array:

```bash
# Basic JSON
ck --json --sem "pattern" src/

# Parse with jq
ck --json --sem "auth" src/ | jq -r '.[].file' | sort -u

# Filter by score
ck --json --sem --scores "pattern" src/ | jq '.[] | select(.score > 0.7)'

# Extract specific fields
ck --json --sem "pattern" src/ | jq '.[] | {file, line, score}'
```

### JSONL Output (Recommended for Agents)

One JSON object per line:

```bash
# JSONL format
ck --jsonl --sem "pattern" src/

# Stream processing
ck --jsonl --sem "pattern" src/ | while read -r line; do
  echo "$line" | jq '.file'
done

# Metadata only (no snippets, smaller output)
ck --jsonl --no-snippet --sem "pattern" src/

# Custom snippet length
ck --jsonl --snippet-length 150 --sem "pattern" src/
```

Why JSONL for AI agents:
- ✅ Stream-friendly: Process results as they arrive
- ✅ Memory-efficient: Parse one result at a time
- ✅ Error-resilient: One malformed line doesn’t break entire response
- ✅ Standard format: Used by OpenAI, Anthropic, modern ML pipelines

## Language Support

### Supported Languages

| Language | Tree-sitter | Semantic Chunking |
|----------|-------------|-------------------|
| Python | ✅ | Functions, classes |
| JavaScript/TypeScript | ✅ | Functions, classes, methods |
| Rust | ✅ | Functions, structs, traits |
| Go | ✅ | Functions, types, methods |
| Ruby | ✅ | Classes, methods, modules |
| Haskell | ✅ | Functions, types, instances |
| C# | ✅ | Classes, interfaces, methods |
| Zig | ✅ | Functions, structs |

Text formats (Markdown, JSON, YAML, TOML, XML, HTML, CSS, shell scripts, SQL) are also supported with content-based chunking.

### Binary Detection

ck uses ripgrep-style content analysis:
- Checks first 8KB of file for NUL bytes
- Automatically indexes text files regardless of extension
- Correctly excludes binary files

## CI/CD Integration

### Exit Codes

```bash
# Exit 0 if matches found, 1 if not
ck "pattern" src/
echo $?  # 0 = found, 1 = not found

# Use in scripts
if ck --hybrid "security issue" src/; then
  echo "Security issues found!"
  exit 1
fi
```

### Pre-commit Hooks

`.git/hooks/pre-commit`:
```bash
#!/bin/bash

# Find TODOs in staged files
if git diff --cached --name-only | xargs ck "TODO|FIXME|XXX"; then
  echo "Warning: Found TODOs in staged files"
  read -p "Commit anyway? (y/n) " -n 1 -r
  echo
  if [[ ! $REPLY =~ ^[Yy]$ ]]; then
    exit 1
  fi
fi

# Check for secrets
if git diff --cached --name-only | xargs ck -i "api_key|password|secret"; then
  echo "Error: Potential secrets found!"
  exit 1
fi
```

### GitHub Actions

```yaml
name: Code Search
on: [pull_request]

jobs:
  search:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install ck
        run: cargo install ck-search

      - name: Search for security issues
        run: |
          ck --hybrid "sql injection|xss|csrf" src/ && exit 1 || true

      - name: Find missing tests
        run: |
          ck -L --sem "test" src/**/*.rs && exit 1 || true
```

## Performance Tuning

### Indexing Performance

```bash
# Use smaller model
ck --index --model bge-small .

# Exclude large directories
ck --exclude "node_modules" --exclude "target" --index .

# Index specific paths only
ck --index src/ lib/ tests/
```

### Search Performance

```bash
# Limit result count
ck --sem --topk 10 "pattern" src/

# Use threshold to reduce computation
ck --sem --threshold 0.5 "pattern" src/

# Search specific directories
ck --sem "pattern" src/core/
```

### Memory Optimization

```bash
# Smaller snippets in JSON output
ck --jsonl --snippet-length 100 --sem "pattern" src/

# Metadata only
ck --jsonl --no-snippet --sem "pattern" src/
```

## Interrupt Handling

All long-running operations can be safely interrupted:

```bash
# Start indexing
ck --index .

# Press Ctrl+C to stop

# Resume later (continues from where it stopped)
ck --index .
```

Partial indexes are saved, and subsequent operations resume from the checkpoint.

## Next Steps

- Deep dive into [semantic search](/features/semantic-search)
- Learn about [MCP integration](/features/mcp-integration)
- Explore [embedding models](/reference/models)
- Check [CLI reference](/reference/cli) for all options


## Links discovered
- [semantic search](https://github.com/BeaconBay/ck/blob/main/features/semantic-search.md)
- [MCP integration](https://github.com/BeaconBay/ck/blob/main/features/mcp-integration.md)
- [embedding models](https://github.com/BeaconBay/ck/blob/main/reference/models.md)
- [CLI reference](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)

--- docs-site/public/AGENTS.md ---
# ck for AI Coding Assistants

Quick reference for AI agents using ck semantic code search.

## Quick Start

```bash
# Index the project (run once at start)
ck --index .

# Semantic search (find by meaning)
ck --sem "error handling" src/

# Check index status
ck --status .
```

## Search Modes

| Mode | Flag | Use When | Example |
|------|------|----------|---------|
| Semantic | `--sem` | Finding concepts, patterns | "retry logic", "input validation" |
| Lexical | `--lex` | Finding exact terms, names | "getUserById", "TODO" |
| Hybrid | `--hybrid` | Balancing precision/recall | "JWT token validation" |
| Regex | `--regex` | Pattern matching | `class.*Handler` |

## Essential Flags

### Threshold (quality control)
```bash
--threshold 0.7   # High-confidence (recommended default)
--threshold 0.5   # Balanced (default if not specified)
--threshold 0.3   # Exploratory (cast wide net)
```

### Result Limiting
```bash
--limit 20        # Recommended default for AI agents
--limit 10        # Quick overview
--limit 50        # Comprehensive analysis
```

### Output Formats
```bash
--jsonl           # Recommended: stream-friendly, one object per line
--json            # Single array (good for small results)
--no-snippet      # Metadata only (faster, smaller)
--snippet-length 150  # Custom snippet size
```

### Show Scores
```bash
--scores          # Display relevance scores with results
```

## Command Patterns

### Codebase Onboarding
```bash
ck --index .
ck --sem "main entry point" .
ck --sem "application initialization" .
ck --sem "authentication" .
ck --sem "database queries" .
```

### Code Search
```bash
# Semantic search with high confidence
ck --sem --threshold 0.7 --limit 20 "error handling" src/

# Lexical search for exact terms
ck --lex "TODO" .

# Hybrid search for balanced results
ck --hybrid --limit 20 "connection pooling" src/

# Get structured output
ck --jsonl --sem --threshold 0.7 "pattern" src/
```

### Code Review
```bash
# Find security patterns
ck --sem "authentication logic" src/
ck --sem "input validation" src/

# Find code quality issues
ck --lex "TODO" .
ck --lex "FIXME" .

# Find performance patterns
ck --sem "caching" src/
ck --sem "database connection" src/
```

### Refactoring
```bash
# Find all implementations of a pattern
ck --sem --threshold 0.8 "user validation" src/

# Find similar code
ck --sem "error response formatting" src/

# Find duplicate logic
ck --sem --threshold 0.8 "duplicate logic" src/
```

## Performance Guidelines

### Optimal Defaults for AI Agents
```bash
ck --sem --threshold 0.7 --limit 20 "query" src/
```

### Index Management
- **Index once** per session at project start: `ck --index .`
- Incremental updates happen automatically
- Only rebuild if index is stale: `ck --clean . && ck --index .`

### Speed Tips
```bash
# Use limit to control result count
ck --sem --limit 10 "query" src/

# Use no-snippet for faster parsing
ck --jsonl --no-snippet --sem "query" src/

# Check index before heavy operations
ck --status .
```

## Common Workflows

### Pattern 1: Understanding New Codebase
```bash
ck --index .
ck --sem "main entry point" .
ck --sem "configuration" .
ck --sem "authentication" .
ck --sem "database" .
ck --sem "API endpoints" .
```

### Pattern 2: Finding Specific Code
```bash
# Use semantic for concepts
ck --sem --threshold 0.7 "JWT authentication" src/

# Use lexical for exact names
ck --lex "authenticateUser" src/

# Use hybrid for balance
ck --hybrid "user session" src/
```

### Pattern 3: Structured Output for Parsing
```bash
# JSONL output (recommended)
ck --jsonl --sem --threshold 0.7 --limit 20 "pattern" src/

# Metadata only (no code snippets)
ck --jsonl --no-snippet --sem "pattern" src/
```

## Troubleshooting

### Irrelevant Results
```bash
# Increase threshold
ck --sem --threshold 0.8 "query" src/

# Switch to hybrid
ck --hybrid "query" src/

# Use lexical for exact terms
ck --lex "ExactFunctionName" src/
```

### Too Few Results
```bash
# Lower threshold
ck --sem --threshold 0.3 "query" src/

# Increase limit
ck --sem --limit 50 "query" src/

# Try hybrid mode
ck --hybrid "query" src/
```

### Slow Searches
```bash
# Limit results
ck --sem --limit 10 "query" src/

# Use metadata only
ck --jsonl --no-snippet --sem "query" src/

# Check index status
ck --status .
```

### Index Out of Date
```bash
ck --status .
ck --index .  # If stale
```

## Model Selection

Default model (`bge-small`) works well for most cases. Switch only if needed:

```bash
# Code-specialized model
ck --switch-model jina-code .

# Large context model
ck --switch-model nomic-v1.5 .
```

## Output Formats Explained

### Human-Readable (default)
```bash
ck --sem "pattern" src/
```

### JSONL (recommended for agents)
```bash
ck --jsonl --sem "pattern" src/
# Output: one JSON object per line
# {"file": "...", "line": 42, "content": "...", "score": 0.847}
```

### JSON
```bash
ck --json --sem "pattern" src/
# Output: single JSON array
# [{"file": "...", "line": 42, ...}, ...]
```

## Best Practices Checklist

✅ **Do**:
- Index once at project start: `ck --index .`
- Use `--threshold 0.7` as default
- Use `--limit 20` to manage context windows
- Use `--jsonl` for structured output
- Use semantic search for concepts
- Use lexical search for exact identifiers

❌ **Don’t**:
- Re-index unnecessarily (automatic incremental updates)
- Use threshold > 0.9 (too restrictive)
- Use threshold < 0.3 (too noisy)
- Skip `--limit` on exploratory searches (overwhelming output)

## Performance Benchmarks

| Operation | Time | Notes |
|-----------|------|-------|
| Initial index (100K LOC) | ~10s | One-time cost |
| Semantic search | <500ms | After initial index |
| Hybrid search | <300ms | Leverages both indices |
| Lexical search | <100ms | Full-text index only |
| Incremental update | <1s | Only changed files |

## MCP Integration

If using ck via MCP protocol (Claude Desktop, etc.):

```bash
# Start MCP server
ck --serve

# Configure in Claude Desktop
claude mcp add ck-search -s user -- ck --serve
```

MCP-specific notes:
- Use pagination for large results (automatic)
- Default page size of 25 is good for most cases
- Use threshold parameter to filter results
- Check index status before heavy searches

## Complete Example Session

```bash
# 1. Index the project
ck --index .

# 2. Check index status
ck --status .

# 3. Find authentication code
ck --sem --threshold 0.7 --limit 20 "authentication" src/

# 4. Get structured output for parsing
ck --jsonl --sem --threshold 0.7 --limit 20 "authentication" src/

# 5. Find all TODOs
ck --lex "TODO" .

# 6. Find error handling patterns
ck --sem --threshold 0.7 "error handling" src/

# 7. Get complete code sections
ck --sem --full-section "retry logic" src/
```

## Quick Command Reference

```bash
# Basic searches
ck --sem "concept" src/                    # Semantic
ck --lex "keyword" src/                    # Lexical
ck --hybrid "query" src/                   # Hybrid
ck --regex "pattern" src/                  # Regex

# With quality control
ck --sem --threshold 0.7 "concept" src/    # High confidence
ck --sem --limit 20 "concept" src/         # Limit results
ck --sem --scores "concept" src/           # Show scores

# Structured output
ck --jsonl --sem "concept" src/            # JSONL format
ck --json --sem "concept" src/             # JSON format
ck --jsonl --no-snippet --sem "concept" src/  # Metadata only

# Index management
ck --index .                               # Build index
ck --status .                              # Check status
ck --clean .                               # Remove index
ck --inspect src/file.rs                   # Inspect chunking

# Advanced
ck --sem --full-section "pattern" src/     # Complete code sections
ck --switch-model jina-code .              # Change embedding model
```

## Additional Resources

For complete documentation, visit: https://beaconbay.github.io/ck/


--- docs-site/guide/ai-agent-setup.md ---
---
title: AI Agent Setup
description: Configure ck for AI coding assistants like Claude Code and Cursor. Best practices for semantic search, JSONL output, and MCP integration with AI agents.
---

# Configuring ck for AI Coding Agents

Best practices for integrating ck with AI coding assistants like Claude Code, Cursor, and other agent-based development tools.

## Overview

AI coding agents benefit significantly from semantic code search capabilities. This guide shows how to configure ck to work optimally with AI assistants, providing them with powerful code understanding and navigation features.

### Benefits of ck for AI Agents

- **Conceptual search** - Agents can find code by describing functionality, not just keywords
- **Context gathering** - Quickly retrieve relevant code sections for agent analysis
- **Codebase understanding** - Help agents map out architecture and patterns
- **Precise results** - Tunable thresholds ensure high-quality matches
- **Efficient output** - Structured formats (JSON/JSONL) for easy parsing

## Claude Code Integration

Claude Code is Anthropic’s official CLI for Claude. Here’s how to configure it for optimal ck usage.

### Tool Permissions

Add ck commands to your allowed tools in `settings.local.json`:

```json
{
  "approvedCommandPatterns": [
    "Bash(ck:*)",
    "Bash(ck --index:*)",
    "Bash(ck --sem:*)",
    "Bash(ck --hybrid:*)",
    "Bash(ck --lex:*)"
  ]
}
```

This allows Claude Code to:
- Run any ck command without prompting
- Index your codebase automatically
- Perform semantic, hybrid, and lexical searches
- Access all ck features seamlessly

### Usage Patterns Documentation

Create a `.claude/` directory in your project with guidance for Claude:

```markdown
# .claude/ck-semantic-search.md

## Hybrid Code Search with ck

Use `ck` for finding code by meaning, not just keywords.

### Search Modes

- `ck --sem "concept"` - Semantic search (by meaning)
- `ck --lex "keyword"` - Lexical search (full-text)
- `ck --hybrid "query"` - Combined regex + semantic
- `ck --regex "pattern"` - Traditional regex search

### Best Practices

::: tip Recommended Usage Patterns
1. **Index once per session**: Run `ck --index .` at project start
2. **Use semantic for concepts**: "error handling", "database queries"
3. **Use lexical for names**: "getUserById", "AuthController"
4. **Tune threshold**: `--threshold 0.7` for high-confidence results
5. **Limit results**: `--limit 20` for focused output
:::

### Example Workflows

# Find authentication logic
ck --sem "user authentication" src/

# Find all TODO comments
ck --lex "TODO" .

# Find error handling patterns with high confidence
ck --sem --threshold 0.8 "error handling" src/
```

### Real-World Example

From [BeaconBay/ck#36](https://github.com/BeaconBay/ck/issues/36):

```bash
# Initial setup
ck --index .

# Semantic search for concepts
ck --sem "error handling" src/
ck --sem "database connection pooling" lib/

# High-confidence matches only
ck --sem --threshold 0.8 "authentication logic" src/

# Limited result set
ck --sem --limit 10 "caching strategy" .
```

## General AI Agent Configuration

These practices apply to any AI coding assistant using ck.

### Index Strategy

**Index once, search many times:**

```bash
# At project start or after significant changes
ck --index .

# Check index status
ck --status .

# Incremental updates happen automatically
# Manual rebuild only if needed
ck --clean . && ck --index .
```

**Why**: Indexing is expensive (1-2 minutes for 1M LOC). Searches are fast (<500ms). Let ck handle incremental updates automatically.

### Search Mode Selection

Choose the right mode for the task:

| Mode | Flag | Best For | Example |
|------|------|----------|---------|
| **Semantic** | `--sem` | Conceptual understanding | "retry logic", "input validation" |
| **Lexical** | `--lex` | Exact terms, names | "AuthService", "TODO", "FIXME" |
| **Hybrid** | `--hybrid` | Balance precision/recall | "JWT token validation" |
| **Regex** | `--regex` | Pattern matching | `class.*Handler`, `fn \w+Error` |

### Performance Tuning

#### Relevance Thresholds

Control result quality with `--threshold`:

```bash
# Exploratory search (cast wide net)
ck --sem --threshold 0.3 "pattern matching" src/

# Balanced search (default)
ck --sem --threshold 0.5 "authentication" src/

# High-confidence only (precise results)
ck --sem --threshold 0.8 "error handling" src/

# Very strict (only closest matches)
ck --sem --threshold 0.9 "exact concept" src/
```

::: tip Threshold Sweet Spot for AI Agents
Start with `0.7` for focused, high-confidence results. If you get too few matches, adjust down to `0.5` for broader coverage.
:::

#### Result Limiting

Prevent overwhelming output:

```bash
# Top 10 results (quick overview)
ck --sem --limit 10 "pattern" src/

# Top 20 results (balanced)
ck --sem --limit 20 "pattern" src/

# Top 50 results (comprehensive)
ck --sem --limit 50 "pattern" src/
```

**Recommendation for AI agents**: Use `--limit 20` by default to keep context windows manageable.

#### Pagination for Large Results

For comprehensive analysis:

```bash
# First page (25 results)
ck --sem --page-size 25 "pattern" src/

# Next page (if needed)
ck --sem --page-size 25 --cursor "abc123" "pattern" src/
```

### Output Formats for AI Consumption

#### JSONL (Recommended)

One JSON object per line - optimal for streaming and parsing:

```bash
# Full output
ck --jsonl --sem "pattern" src/

# Metadata only (smaller, faster)
ck --jsonl --no-snippet --sem "pattern" src/

# Custom snippet length
ck --jsonl --snippet-length 150 --sem "pattern" src/
```

::: tip Why JSONL for AI Agents
- **Stream-friendly**: Process results as they arrive
- **Memory-efficient**: Parse one result at a time
- **Error-resilient**: One malformed line doesn't break everything
- **Standard format**: Used by OpenAI, Anthropic, modern ML pipelines
:::

#### JSON

Single array - good for small result sets:

```bash
ck --json --sem "pattern" src/ | jq '.[].file' | sort -u
```

### Embedding Model Selection

Choose the right model for your use case:

| Model | Best For AI Agents | Trade-offs |
|-------|-------------------|------------|
| **bge-small** (default) | Quick searches, fast indexing | Smaller chunks (400 tokens) |
| **nomic-v1.5** | Large functions, documentation | Larger download (~500MB) |
| **jina-code** | Code-specialized understanding | Larger download (~500MB) |

**Recommendation**:
- Start with `bge-small` (default) - works well for most codebases
- Switch to `jina-code` if agent struggles with code-specific concepts
- Use `nomic-v1.5` for documentation-heavy projects

```bash
# Switch models
ck --switch-model jina-code .
```

## Recommended .ckignore for AI Agents

Optimize indexing for AI-assisted development:

```txt
# .ckignore

# Build artifacts (not useful for understanding code)
target/
dist/
build/
*.o
*.so
*.dylib

# Dependencies (too large, low signal)
node_modules/
vendor/
venv/
.venv/

# Media files (not code)
*.png
*.jpg
*.gif
*.mp4
*.pdf

# Large data files
*.csv
*.json
*.xml
*.log

# Test fixtures (unless you search them)
fixtures/
__snapshots__/
*.snap

# Generated code (if not relevant)
*_pb2.py
*.generated.*

# Documentation (include if you want AI to reference docs)
# docs/
# *.md
```

**Key principle**: Exclude anything that doesn’t help the AI understand your codebase’s logic and architecture.

## Common Workflows

### Codebase Onboarding

Help AI agents understand new projects:

```bash
# Index the project
ck --index .

# Find entry points
ck --sem "main entry point" .
ck --sem "application initialization" .

# Understand architecture
ck --sem "dependency injection" src/
ck --sem "data flow" src/

# Map out key components
ck --sem "authentication" .
ck --sem "database queries" .
ck --sem "API endpoints" .
```

### Code Review Assistance

Find code patterns for review:

```bash
# Security checks
ck --sem "authentication logic" src/
ck --sem "input validation" src/
ck --sem "SQL queries" src/

# Code quality
ck --lex "TODO" .
ck --lex "FIXME" .
ck --sem "error handling" src/

# Performance
ck --sem "caching" src/
ck --sem "database connection" src/
```

### Refactoring Support

Find code to refactor:

```bash
# Find all implementations of a pattern
ck --sem "user validation" src/

# Find similar code for consistency
ck --sem "error response formatting" src/

# Find candidates for extraction
ck --sem --threshold 0.8 "duplicate logic" src/
```

### Documentation Generation

Gather code for documentation:

```bash
# Find public APIs
ck --sem "public api" src/

# Find configuration
ck --sem "configuration options" .

# Extract examples
ck --sem "example usage" tests/
```

## MCP Integration

For MCP-compatible AI agents (Claude Desktop, etc.), use the MCP server:

### Setup

```bash
# Start MCP server
ck --serve

# Configure in Claude Desktop
claude mcp add ck-search -s user -- ck --serve
```

### MCP-Specific Best Practices

When using ck via MCP protocol:

1. **Use pagination** - Large result sets are paginated automatically
2. **Set reasonable page_size** - Default 25 is good for most cases
3. **Use threshold parameter** - Filter results for relevance
4. **Check index status** - Call `index_status` tool before heavy searches
5. **Reindex after bulk changes** - Call `reindex` tool after git pull

See [MCP Integration](/features/mcp-integration) for complete API reference.

## Troubleshooting

### Agent Gets Irrelevant Results

**Solution 1**: Increase threshold

```bash
ck --sem --threshold 0.8 "your query" src/
```

**Solution 2**: Switch to hybrid search

```bash
ck --hybrid "your query" src/
```

**Solution 3**: Try a different model

```bash
ck --switch-model jina-code .
```

### Agent Searches Take Too Long

**Solution 1**: Limit results

```bash
ck --sem --limit 10 "query" src/
```

**Solution 2**: Exclude large directories

```bash
# Add to .ckignore
node_modules/
vendor/
dist/
```

**Solution 3**: Use metadata-only output

```bash
ck --jsonl --no-snippet --sem "query" src/
```

### Agent Misses Obvious Matches

**Solution 1**: Lower threshold

```bash
ck --sem --threshold 0.3 "query" src/
```

**Solution 2**: Use hybrid search

```bash
ck --hybrid "query" src/
```

**Solution 3**: Use lexical search for exact terms

```bash
ck --lex "ExactFunctionName" src/
```

### Near-Miss Threshold Hints

**Feature**: When no results are found above the specified threshold, ck provides helpful "near-miss" hints.

**How it helps AI agents**:
When semantic search finds no matches above your threshold, ck will show the closest match that fell just below the threshold, along with its score. This gives AI agents a clear signal to adjust the threshold parameter intelligently.

**Example output**:
```bash
$ ck --sem --threshold 0.7 "retry logic" src/

No matches found above threshold 0.7

Near-miss:
  src/network/client.rs:156 (score: 0.68)
  "Implements exponential backoff for failed requests"

Hint: Try lowering threshold to 0.65 or use --threshold 0.6
```

**AI agent workflow**:
1. Initial search with `--threshold 0.7` returns no results
2. ck shows near-miss at 0.68
3. Agent automatically retries with `--threshold 0.65`
4. Finds relevant matches

This feedback loop helps agents converge on optimal thresholds without overshooting.

::: tip Why This Matters for AI Agents
AI agents naturally understand threshold tuning. Near-miss hints provide actionable feedback, allowing agents to adjust parameters intelligently rather than guessing or giving up. This is one of ck's "nice affordances" for AI-first usage.
:::

### Index Out of Date

**Solution**: Check status and reindex if needed

```bash
ck --status .
# If stale:
ck --index .
```

## Performance Benchmarks

Typical performance for AI agent workflows:

| Operation | Time | Notes |
|-----------|------|-------|
| Initial index (100K LOC) | ~10s | One-time cost |
| Semantic search | <500ms | Cached index |
| Hybrid search | <300ms | Leverages both indices |
| Lexical search | <100ms | Full-text index |
| Incremental update | <1s | Only changed files |

## Best Practices Summary

✅ **Do**:
- Index once at project start
- Use `--threshold 0.7` as default for AI agents
- Use `--limit 20` to keep context manageable
- Prefer JSONL output for parsing
- Use semantic search for concepts
- Use lexical search for exact identifiers
- Tune thresholds based on result quality

❌ **Don’t**:
- Re-index unnecessarily (ck handles incremental updates)
- Use threshold above 0.9 (too restrictive)
- Use threshold below 0.3 (too noisy)
- Omit `--limit` for exploratory searches (too many results)
- Index huge binary/media/data files
- Mix search modes without understanding differences

## Next Steps

- Read [MCP Integration](/features/mcp-integration) for API details
- See [Choosing an Interface](/guide/choosing-interface) for workflow comparison
- Check [Semantic Search](/features/semantic-search) for search fundamentals
- Review [Advanced Usage](/guide/advanced-usage) for power-user techniques
- Explore [CLI Reference](/reference/cli) for all flags and options


## Links discovered
- [BeaconBay/ck#36](https://github.com/BeaconBay/ck/issues/36)
- [MCP Integration](https://github.com/BeaconBay/ck/blob/main/features/mcp-integration.md)
- [Choosing an Interface](https://github.com/BeaconBay/ck/blob/main/guide/choosing-interface.md)
- [Semantic Search](https://github.com/BeaconBay/ck/blob/main/features/semantic-search.md)
- [Advanced Usage](https://github.com/BeaconBay/ck/blob/main/guide/advanced-usage.md)
- [CLI Reference](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)

--- docs-site/recipes/ai-workflows.md ---
---
title: AI Agent Search Workflows
description: Integrate ck with AI agents for autonomous code analysis. MCP server setup, LLM tool integration, and workflow patterns for Claude, ChatGPT, and custom agents.
---

# AI Agent Search Workflows

**Goal**: Integrate ck with AI agents to enable autonomous code search, analysis, and understanding.

**Difficulty**: Intermediate

::: tip AI-First Design
ck was designed **primarily for AI agents**, not humans. AI agents naturally understand how to tune `--threshold`, `--topk`, and interpret semantic scores. The TUI exists to help humans gain confidence, but the CLI is where ck shines with AI.
:::

## Why ck for AI Agents?

### Traditional Tools

- **grep**: Finds exact matches only
- **cscope/ctags**: Symbol-based, requires exact names
- **AST parsers**: Complex, language-specific

### ck Advantages for AI

1. **Semantic understanding**: Find code by meaning, not just keywords
2. **JSON output**: Perfect for LLM consumption
3. **Adaptive search**: AI agents naturally tune parameters
4. **Near-miss hints**: Guides threshold adjustment
5. **grep compatibility**: Familiar tool AI models know well

## Integration Methods

### Method 1: Model Context Protocol (MCP)

**Best for**: Claude Desktop, Claude Code, or MCP-compatible tools

See the complete MCP setup guide: [MCP Integration](/features/mcp-integration)

**Quick start**:

```json
{
  "mcpServers": {
    "ck": {
      "command": "ck",
      "args": ["--mcp"],
      "cwd": "/path/to/your/project"
    }
  }
}
```

**Capabilities**:

- `search_semantic`: Semantic code search
- `search_hybrid`: Hybrid semantic + keyword
- `search_keyword`: Traditional grep
- `list_indexed_files`: Show indexed files
- `get_file_chunks`: Get semantic chunks from file

### Method 2: Direct CLI Integration

**Best for**: Custom agents, automation scripts, LangChain tools

**Tool definition** (JSON Schema):

```json
{
  "name": "search_code_semantic",
  "description": "Search codebase using semantic understanding. Finds code by meaning, not just exact matches. Returns file paths, line numbers, code snippets, and relevance scores.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "Natural language description of what to find. Examples: 'error handling', 'database connection', 'user authentication'"
      },
      "threshold": {
        "type": "number",
        "description": "Minimum relevance score (0.0-1.0). Default 0.6. Lower = more results, higher = more precise.",
        "default": 0.6
      },
      "limit": {
        "type": "number",
        "description": "Maximum number of results to return. Default 10, max 100.",
        "default": 10
      }
    },
    "required": ["query"]
  }
}
```

**Implementation**:

```python
import subprocess
import json

def search_code_semantic(
    query: str,
    threshold: float = 0.6,
    limit: int = 10
) -> list:
    """Search codebase semantically."""
    cmd = [
        "ck",
        "--sem", query,
        ".",
        "--json",
        "--scores",
        "--threshold", str(threshold),
        "--limit", str(limit)
    ]

    result = subprocess.run(cmd, capture_output=True, text=True)

    if result.returncode != 0:
        raise RuntimeError(f"Search failed: {result.stderr}")

    return json.loads(result.stdout)
```

### Method 3: LangChain Integration

**Best for**: LangChain-based applications

```python
from langchain.tools import Tool
from langchain.agents import initialize_agent, AgentType
from langchain.llms import OpenAI
import subprocess
import json

def ck_search(query: str) -> str:
    """Execute ck semantic search and format results."""
    cmd = ["ck", "--sem", query, ".", "--json", "--limit", "10", "--scores"]
    result = subprocess.run(cmd, capture_output=True, text=True)

    if result.returncode != 0:
        return f"Search failed: {result.stderr}"

    results = json.loads(result.stdout)

    # Format for LLM
    formatted = []
    for r in results:
        formatted.append(
            f"File: {r['file']}:{r['line']}\n"
            f"Score: {r['score']:.3f}\n"
            f"Code: {r['content']}\n"
        )

    return "\n---\n".join(formatted)

# Create LangChain tool
ck_tool = Tool(
    name="SearchCode",
    func=ck_search,
    description=(
        "Search codebase semantically by describing what you're looking for. "
        "Input should be a natural language description like 'error handling' "
        "or 'database connection logic'. Returns relevant code with file locations."
    )
)

# Initialize agent with ck tool
llm = OpenAI(temperature=0)
agent = initialize_agent(
    tools=[ck_tool],
    llm=llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True
)

# Agent can now search code
response = agent.run("Find the authentication logic in this codebase")
```

## AI Agent Workflow Patterns

### Pattern 1: Iterative Refinement

AI agents naturally refine searches based on results:

```python
# Initial broad search
results = search_code_semantic("authentication", threshold=0.6, limit=10)

if len(results) == 0:
    # No results - lower threshold
    results = search_code_semantic("authentication", threshold=0.4, limit=10)

elif len(results) > 50:
    # Too many results - increase threshold
    results = search_code_semantic("authentication", threshold=0.75, limit=10)

# Found key function, now find usages
if results:
    func_name = extract_function_name(results[0]['content'])
    # Switch to hybrid for exact matching
    usages = search_code_hybrid(func_name, threshold=0.02, limit=50)
```

### Pattern 2: Multi-Stage Analysis

Break complex tasks into stages:

```python
# Stage 1: Find entry point
entry_points = search_code_semantic("main application entry point", limit=5)

# Stage 2: Discover architecture
architecture = search_code_semantic("service layer dependency injection", limit=15)

# Stage 3: Find database code
db_code = search_code_semantic("database queries repository", limit=20)

# Stage 4: Locate error handling
error_handling = search_code_semantic("error handling custom errors", limit=15)

# Synthesize findings
report = synthesize_analysis(entry_points, architecture, db_code, error_handling)
```

### Pattern 3: Security Audit Workflow

```python
async def security_audit(project_path: str) -> dict:
    """Comprehensive AI-driven security audit."""

    findings = {}

    # 1. Authentication
    findings['auth'] = await analyze_security(
        search_code_semantic("authentication login password", limit=20),
        check_for=["weak_hashing", "hardcoded_secrets", "no_rate_limit"]
    )

    # 2. SQL Injection
    findings['sql_injection'] = await analyze_security(
        search_code_semantic("sql query user input", limit=30),
        check_for=["string_concatenation", "no_parameterization"]
    )

    # 3. XSS
    findings['xss'] = await analyze_security(
        search_code_semantic("html render user content", limit=25),
        check_for=["no_escaping", "dangerouslySetInnerHTML"]
    )

    # 4. Hardcoded secrets
    findings['secrets'] = await analyze_security(
        search_code_semantic("secret password api key hardcoded", limit=15),
        check_for=["credentials_in_code"]
    )

    return findings
```

### Pattern 4: Code Understanding Assistant

```python
def understand_codebase(question: str, context: str = "") -> str:
    """Answer questions about codebase using ck."""

    # Parse question to determine search strategy
    if is_asking_about_specific_function(question):
        # Use hybrid search for known names
        query = extract_function_name(question)
        results = search_code_hybrid(query, limit=10)

    elif is_asking_about_concept(question):
        # Use semantic search for concepts
        query = extract_concept(question)
        results = search_code_semantic(query, limit=15)

    else:
        # General exploration
        query = question
        results = search_code_semantic(query, limit=20)

    # Provide results to LLM for synthesis
    return synthesize_answer(question, results, context)
```

## Real-World Examples

### Example 1: Code Review Assistant

```python
from anthropic import Anthropic
import json
import subprocess

client = Anthropic(api_key="...")

def get_code_context(query: str, limit: int = 10) -> str:
    """Get relevant code context using ck."""
    cmd = ["ck", "--sem", query, ".", "--json", "--limit", str(limit), "--scores"]
    result = subprocess.run(cmd, capture_output=True, text=True)
    results = json.loads(result.stdout)

    context = []
    for r in results:
        context.append(f"# {r['file']}:{r['line']} (score: {r['score']:.2f})\n{r['content']}")

    return "\n\n".join(context)

def review_pull_request(pr_description: str, changed_files: list) -> str:
    """AI code review using ck for context."""

    # Get relevant context from codebase
    context_queries = [
        "error handling patterns",
        "authentication authorization",
        "database queries",
        "input validation"
    ]

    context = ""
    for query in context_queries:
        context += f"\n## {query}\n"
        context += get_code_context(query, limit=5)

    # Ask Claude to review
    message = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=4096,
        messages=[{
            "role": "user",
            "content": f"""Review this pull request for security and quality issues.

PR Description:
{pr_description}

Changed Files:
{json.dumps(changed_files, indent=2)}

Codebase Context (from semantic search):
{context}

Provide:
1. Security concerns
2. Code quality issues
3. Suggestions for improvement
"""
        }]
    )

    return message.content[0].text
```

### Example 2: Documentation Generator

```python
def generate_documentation(component: str) -> str:
    """Generate documentation by understanding code."""

    # Find the component
    component_code = search_code_hybrid(component, limit=5)

    if not component_code:
        return f"Component '{component}' not found"

    # Find related code
    related_queries = [
        f"{component} usage examples",
        f"{component} configuration",
        f"{component} error handling",
        f"{component} tests"
    ]

    related_code = {}
    for query in related_queries:
        related_code[query] = search_code_semantic(query, limit=5)

    # Generate documentation using LLM
    return generate_docs_from_code(component_code, related_code)
```

### Example 3: Refactoring Assistant

```python
def find_refactoring_opportunities(pattern: str) -> dict:
    """Find code duplication and refactoring opportunities."""

    # Find all instances of pattern
    instances = search_code_semantic(pattern, threshold=0.65, limit=50)

    # Group by similarity
    groups = cluster_by_similarity(instances)

    # For each group, suggest refactoring
    suggestions = []
    for group in groups:
        if len(group) >= 3:  # Pattern repeated 3+ times
            suggestion = {
                "pattern": pattern,
                "instances": len(group),
                "locations": [f"{r['file']}:{r['line']}" for r in group],
                "recommendation": suggest_refactoring(group)
            }
            suggestions.append(suggestion)

    return {
        "pattern": pattern,
        "total_instances": len(instances),
        "refactoring_suggestions": suggestions
    }
```

## Handling ck Output

### Parsing JSON Output

````python
import json

def parse_ck_results(output: str) -> list:
    """Parse ck JSON output safely."""
    try:
        results = json.loads(output)
        return results
    except json.JSONDecodeError as e:
        # Handle empty results or errors
        return []

def format_for_llm(results: list, max_length: int = 4000) -> str:
    """Format results for LLM consumption."""
    formatted = []

    for r in results:
        entry = f"""
File: {r['file']}:{r['line']}
{f"Score: {r['score']:.3f}" if 'score' in r else ""}
```
{r.get('content', '')}
```
"""
        formatted.append(entry)

    full_text = "\n---\n".join(formatted)

    # Truncate if too long
    if len(full_text) > max_length:
        full_text = full_text[:max_length] + "\n\n[... truncated ...]"

    return full_text
````

### Handling Near-Miss Feedback

ck provides near-miss hints when no results are found:

```text
No matches found above threshold 0.7
Near-miss: ./auth.rs:42 (score: 0.68)
Hint: Try lowering threshold to 0.6
```

**AI agent pattern**:

```python
def adaptive_search(query: str, threshold: float = 0.6) -> list:
    """Search with adaptive threshold based on results."""
    cmd = ["ck", "--sem", query, ".", "--json", "--threshold", str(threshold), "--limit", "10"]
    result = subprocess.run(cmd, capture_output=True, text=True)

    # Check for near-miss hint in stderr
    if "Near-miss" in result.stderr and "Try lowering threshold to" in result.stderr:
        # Extract suggested threshold
        suggested = extract_threshold_hint(result.stderr)
        # Retry with suggested threshold
        return adaptive_search(query, suggested)

    return json.loads(result.stdout) if result.stdout else []
```

## AI Agent Best Practices

### 1. Start with Semantic, Refine with Hybrid

```python
# 1. Semantic search to discover
results = search_code_semantic("authentication", limit=10)

# 2. Extract key identifiers
identifiers = extract_identifiers(results)  # e.g., "authenticateUser"

# 3. Hybrid search for exact usage
for ident in identifiers:
    usages = search_code_hybrid(ident, limit=30)
```

### 2. Use Scores to Prioritize

```python
results = search_code_semantic("security vulnerability", limit=30)

# Sort by score (already sorted by ck)
high_confidence = [r for r in results if r['score'] > 0.8]
medium_confidence = [r for r in results if 0.6 <= r['score'] <= 0.8]

# Review high confidence first
analyze(high_confidence)
```

### 3. Combine Multiple Searches

```python
# Comprehensive understanding
auth_results = search_code_semantic("authentication login", limit=15)
session_results = search_code_semantic("session management", limit=10)
token_results = search_code_semantic("token jwt verification", limit=10)

# Combine and deduplicate
all_results = deduplicate_by_file_line(
    auth_results + session_results + token_results
)
```

### 4. Index Once Per Session

```python
import os
from pathlib import Path

def ensure_indexed(project_path: str) -> bool:
    """Check if index exists, create if needed."""
    ck_dir = Path(project_path) / ".ck"

    if not ck_dir.exists():
        # Index for first time
        cmd = ["ck", "--index", project_path]
        result = subprocess.run(cmd, capture_output=True)
        return result.returncode == 0

    return True  # Already indexed

# At start of AI session
ensure_indexed("/path/to/project")
```

### 5. Provide Context to LLM

```python
def query_with_context(llm_query: str) -> str:
    """Provide search results as context to LLM."""

    # Extract search intent from LLM query
    search_query = extract_search_query(llm_query)

    # Get relevant code
    code_results = search_code_semantic(search_query, limit=15)

    # Format as context
    context = format_for_llm(code_results)

    # Provide to LLM
    prompt = f"""
Based on this code from the project:

{context}

Answer the following question:
{llm_query}
"""

    return call_llm(prompt)
```

## Performance Considerations

### Token Efficiency

```python
# Instead of sending all code
results = search_code_semantic("auth", limit=50)  # 50 results * 200 chars = 10,000 chars

# Send summary + top results
summary = summarize_results(results)
top_results = results[:10]

context = f"{summary}\n\nTop results:\n{format_for_llm(top_results)}"
# Much more token-efficient
```

### Caching Results

```python
from functools import lru_cache
import hashlib

@lru_cache(maxsize=128)
def cached_search(query: str, threshold: float, limit: int) -> str:
    """Cache search results to avoid repeated calls."""
    cmd = ["ck", "--sem", query, ".", "--json", "--threshold", str(threshold), "--limit", str(limit)]
    result = subprocess.run(cmd, capture_output=True, text=True)
    return result.stdout

# Use cached search
results1 = json.loads(cached_search("authentication", 0.6, 10))  # Executes ck
results2 = json.loads(cached_search("authentication", 0.6, 10))  # Returns cached
```

## Error Handling

```python
def robust_search(query: str, **kwargs) -> list:
    """Search with comprehensive error handling."""
    try:
        cmd = ["ck", "--sem", query, ".", "--json"]

        for key, value in kwargs.items():
            cmd.extend([f"--{key}", str(value)])

        result = subprocess.run(
            cmd,
            capture_output=True,
            text=True,
            timeout=30  # 30 second timeout
        )

        if result.returncode != 0:
            error_msg = result.stderr
            if "not indexed" in error_msg:
                # Try to index
                subprocess.run(["ck", "--index", "."], check=True)
                # Retry search
                return robust_search(query, **kwargs)
            else:
                raise RuntimeError(f"Search failed: {error_msg}")

        return json.loads(result.stdout) if result.stdout else []

    except subprocess.TimeoutExpired:
        return []  # Search took too long, return empty
    except json.JSONDecodeError:
        return []  # Invalid JSON, return empty
    except Exception as e:
        print(f"Search error: {e}")
        return []
```

## Next Steps

- **Learn MCP integration**: [MCP Integration](/features/mcp-integration)
- **Understand semantic search**: [Semantic Search](/features/semantic-search)
- **Explore use cases**: [Exploring New Codebases](/recipes/explore-codebase)
- **Security workflows**: [Security Code Review](/recipes/security-review)

## Related Documentation

- [AI Agent Setup Guide](/guide/ai-agent-setup) - Quick setup for AI usage
- [Output Formats](/reference/output-formats) - JSON schema reference
- [CLI Reference](/reference/cli) - All command-line options
- [MCP Integration](/features/mcp-integration) - Model Context Protocol setup


## Links discovered
- [MCP Integration](https://github.com/BeaconBay/ck/blob/main/features/mcp-integration.md)
- [Semantic Search](https://github.com/BeaconBay/ck/blob/main/features/semantic-search.md)
- [Exploring New Codebases](https://github.com/BeaconBay/ck/blob/main/recipes/explore-codebase.md)
- [Security Code Review](https://github.com/BeaconBay/ck/blob/main/recipes/security-review.md)
- [AI Agent Setup Guide](https://github.com/BeaconBay/ck/blob/main/guide/ai-agent-setup.md)
- [Output Formats](https://github.com/BeaconBay/ck/blob/main/reference/output-formats.md)
- [CLI Reference](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)

--- docs-site/reference/architecture.md ---
---
title: Architecture
description: Technical architecture of ck's modular Rust workspace. Understand crate organization, indexing pipeline, search algorithms, and design decisions.
---

# Architecture

Understanding ck's modular Rust workspace architecture.

## Workspace Structure

ck uses a Cargo workspace with specialized crates:

```
ck/
├── ck-cli/          # Command-line interface and MCP server
├── ck-core/         # Shared types, configuration, utilities
├── ck-engine/       # Search engine implementations
├── ck-index/        # File indexing and sidecar management
├── ck-embed/        # Text embedding providers
├── ck-ann/          # Approximate nearest neighbor indices
├── ck-chunk/        # Text segmentation and parsing
└── ck-models/       # Model registry and configuration
```

## Crate Responsibilities

### ck-cli

**Purpose**: User-facing CLI and MCP server

**Key components:**
- Argument parsing (clap)
- MCP JSON-RPC server
- Output formatting
- User interaction

**Dependencies:** All other crates

### ck-core

**Purpose**: Shared types and utilities

**Key components:**
- SearchResult types
- Configuration structures
- Error types (anyhow)
- Common utilities

**Dependencies:** None (foundation crate)

### ck-engine

**Purpose**: Search implementations

**Key components:**
- RegexEngine: Pattern matching
- SemanticEngine: Vector similarity search
- HybridEngine: Reciprocal Rank Fusion
- Result ranking and scoring

**Dependencies:** ck-core, ck-index, ck-embed, ck-ann

### ck-index

**Purpose**: File indexing and management

**Key components:**
- File discovery and traversal
- Hash-based change detection
- Incremental index updates
- Sidecar file management
- Exclusion pattern handling

**Dependencies:** ck-core, ck-chunk

### ck-embed

**Purpose**: Embedding generation

**Key components:**
- FastEmbed integration
- Multiple model support (BGE, Nomic, Jina)
- Token-aware chunking
- Embedding caching
- Model download management

**Dependencies:** ck-core, ck-models

### ck-ann

**Purpose**: Vector similarity search

**Key components:**
- Approximate Nearest Neighbor indices
- Cosine similarity scoring
- Index persistence
- Vector storage

**Dependencies:** ck-core

### ck-chunk

**Purpose**: Intelligent code chunking

**Key components:**
- Tree-sitter parsing (7+ languages)
- Semantic boundary detection
- Token counting (HuggingFace tokenizers)
- Content-based text detection
- Language detection

**Dependencies:** ck-core, ck-models

### ck-models

**Purpose**: Model configuration

**Key components:**
- Model registry (BGE, Nomic, Jina)
- Token limits and dimensions
- Model aliases
- Chunking configuration

**Dependencies:** ck-core

## Data Flow

### Indexing Flow

```
User Command (ck --index .)
    ↓
ck-cli: Parse arguments
    ↓
ck-index: Discover files
    ↓
ck-chunk: Parse and segment code
    ↓
ck-embed: Generate embeddings
    ↓
ck-ann: Build vector index
    ↓
ck-index: Save index to .ck/
```

### Search Flow (Semantic)

```
User Query (ck --sem "pattern" .)
    ↓
ck-cli: Parse arguments
    ↓
ck-embed: Embed query
    ↓
ck-ann: Find similar vectors
    ↓
ck-engine: Rank and score results
    ↓
ck-cli: Format and display output
```

### Search Flow (Hybrid)

```
User Query (ck --hybrid "pattern" .)
    ↓
ck-cli: Parse arguments
    ↓
[Parallel]
├─ ck-engine (SemanticEngine): Semantic search
└─ ck-engine (RegexEngine): Keyword search
    ↓
ck-engine (HybridEngine): RRF fusion
    ↓
ck-cli: Format and display output
```

## Key Design Patterns

### Error Handling

Uses `anyhow::Result` consistently:

```rust
use anyhow::Result;

pub fn search(query: &str) -> Result<Vec<SearchResult>> {
    // ...
}
```

### Async/Await

Tokio runtime for I/O operations:

```rust
#[tokio::main]
async fn main() -> Result<()> {
    // ...
}
```

### Parallel Processing

Rayon for CPU-intensive tasks:

```rust
use rayon::prelude::*;

files.par_iter()
    .map(|f| process_file(f))
    .collect()
```

### Memory-Mapped Files

Efficient large file access:

```rust
use memmap2::Mmap;

let mmap = unsafe { Mmap::map(&file)? };
// Access file contents without full load
```

## Storage Format

### Index Structure

```
.ck/
├── manifest.json          # Index metadata
│   └── { model, dimensions, timestamp, ... }
├── embeddings.json        # Vector embeddings
│   └── { file_path: [vectors...], ... }
├── ann_index.bin          # ANN index (binary)
└── tantivy_index/         # Keyword search index
    ├── meta.json
    └── *.seg files
```

### Sidecar Files

Each source file gets a sidecar:

```
src/
├── main.rs
└── .ck/
    └── main.rs.ck         # Sidecar with chunks and hashes
```

Sidecar contains:
- File hash (for change detection)
- Chunk boundaries
- Embedding IDs
- Metadata

## Performance Considerations

### Indexing Performance

- **Parallel file processing** – Rayon thread pool
- **Incremental updates** – Hash-based change detection
- **Efficient I/O** – Memory-mapped files
- **Smart exclusions** – Early filtering of non-code files

### Search Performance

- **Vector search** – O(log n) with ANN index
- **Keyword search** – Tantivy inverted index
- **Caching** – Embedding cache, model cache
- **Streaming results** – Generator patterns for large result sets

### Memory Management

- **Lazy loading** – Files loaded only when needed
- **Streaming processing** – Process files one at a time
- **Index compression** – Binary format for vectors
- **Model caching** – Reuse loaded models

## Testing Strategy

### Unit Tests

Each crate has unit tests:

```bash
cargo test --workspace
```

### Integration Tests

End-to-end testing in ck-cli:

```bash
cargo test --package ck-cli
```

### Feature Tests

Test each feature combination:

```bash
cargo hack test --each-feature --workspace
```

## Build Process

### Development

```bash
# Build all crates
cargo build --workspace

# Build release
cargo build --workspace --release

# Run tests
cargo test --workspace

# Lint
cargo clippy --workspace --all-features

# Format
cargo fmt --all
```

### Release

```bash
# Version bump (all crates)
# Update Cargo.toml in each crate

# Build and test
cargo test --workspace
cargo clippy --workspace --all-features
cargo fmt --all --check

# Publish to crates.io
cargo publish --package ck-core
cargo publish --package ck-models
# ... (publish in dependency order)
cargo publish --package ck-cli
```

## Extension Points

### Adding New Embedding Models

1. Add model config to `ck-models/src/registry.rs`
2. Implement embedding in `ck-embed`
3. Add CLI flag support in `ck-cli`

### Adding New Languages

1. Add tree-sitter grammar to `ck-chunk/Cargo.toml`
2. Implement parser in `ck-chunk/src/parsers/`
3. Register language in `ck-chunk/src/lib.rs`

### Adding New Search Modes

1. Implement engine in `ck-engine/src/`
2. Add CLI flag in `ck-cli/src/args.rs`
3. Wire up in `ck-cli/src/main.rs`

## Next Steps

- Read [contributing guide](/contributing/development)
- Check [CLI reference](/reference/cli)
- Explore [embedding models](/reference/models)
- See [configuration](/reference/configuration)


## Links discovered
- [contributing guide](https://github.com/BeaconBay/ck/blob/main/contributing/development.md)
- [CLI reference](https://github.com/BeaconBay/ck/blob/main/reference/cli.md)
- [embedding models](https://github.com/BeaconBay/ck/blob/main/reference/models.md)
- [configuration](https://github.com/BeaconBay/ck/blob/main/reference/configuration.md)

--- examples/README.md ---
# ck Examples

This folder demonstrates practical use cases for `ck` (semantic grep) with real code examples.

## Quick Start

1. **Index the examples first** (required for semantic/lexical/hybrid search):
   ```bash
   ck --index examples/
   ```

2. **Try the quick demo**:
   ```bash
   ./examples/quick_demo.sh
   ```

3. **Try the examples below** using the demo files in this folder.

## Demo Files

### Code Examples (`code/`)
- **`full_section_demo.py`** - Python code with classes, functions, error handling  
- **`web_server.rs`** - Rust web server with authentication, database connections, error handling
- **`api_client.js`** - JavaScript API client with HTTP requests, retry logic, authentication

### Text Samples (`text_samples/`)
- **`fixtures/`** - Simple demo text files (demo1.txt to demo10.txt)
- **`wiki_articles/`** - Wikipedia articles on various topics (AI, science, technology, etc.)

## Search Mode Examples

### 1. **Regex Search** (Default - no indexing required)
Classic grep-style pattern matching:

```bash
# Basic text search
ck "error" examples/

# Case-insensitive search  
ck -i "database" examples/

# Whole word matching
ck -w "class" examples/

# Fixed string (no regex)
ck -F "def __init__" examples/

# With line numbers and context
ck -n -C 2 "error" examples/
```

### 2. **Semantic Search** (Finds conceptually similar code)
Understands meaning, not just text matches:

```bash
# Find error handling patterns (try/catch, exception handling, etc.)
ck --sem "error handling" examples/

# Find database-related code
ck --sem "database connection" text_samples/

# Find authentication logic
ck --sem "user authentication" examples/

# Limit results and show scores
ck --sem --limit 5 --scores "data processing" examples/

# Higher precision filtering
ck --sem --threshold 0.8 "function" examples/
```

### 3. **Lexical Search** (BM25 full-text search with ranking)
Better than regex for phrase matching:

```bash
# Full-text search with relevance ranking
ck --lex "user authentication" examples/

# Multi-word phrases
ck --lex "error handling patterns" examples/
```

### 4. **Hybrid Search** (Best of both worlds)
Combines regex precision with semantic understanding:

```bash
# Most comprehensive search
ck --hybrid "async function" examples/

# Find specific patterns with semantic context
ck --hybrid "database" --limit 10 examples/

# Show relevance scores
ck --hybrid "error" --scores examples/
```

## Advanced Features

### Full Section Mode
Get complete code blocks instead of just matching lines:

```bash
# Return entire functions/classes containing matches
ck --sem --full-section "error handling" examples/

# Works with all search modes
ck --hybrid --full-section "database" examples/
```

### JSON Output
Machine-readable results for scripts and tools:

```bash
# JSON output for integration
ck --json --sem "error handling" examples/

# Pipe to jq for processing
ck --json --sem "database" examples/ | jq '.preview'
```

### Context and Formatting
```bash
# Show surrounding lines
ck -A 3 -B 1 "class" examples/    # 3 after, 1 before
ck -C 2 "def" examples/           # 2 lines of context

# File listing modes
ck -l "error" examples/           # List files with matches
ck -L "nonexistent" examples/     # List files without matches
```

## Practical Use Cases

### 1. **Code Review**
```bash
# Find potential security issues
ck --sem "sql injection" examples/
ck --sem "password storage" examples/

# Find error handling patterns
ck --sem "exception handling" examples/
```

### 2. **Code Exploration** 
```bash
# Understand new codebase
ck --sem "main entry point" examples/
ck --sem "configuration setup" examples/

# Find similar implementations
ck --sem "data validation" examples/
```

### 3. **Refactoring**
```bash
# Find deprecated patterns
ck --hybrid "old api" examples/

# Locate specific implementations
ck --sem "async operations" examples/
```

### 4. **Documentation**
```bash
# Find examples of usage
ck --sem "example usage" examples/

# Locate API patterns
ck --sem "rest api" examples/
```

## Performance Tips

- **Index once, search many times**: Use `ck --index` before semantic searches
- **Use appropriate search modes**: 
  - Regex for exact patterns
  - Semantic for concept-based searches  
  - Hybrid for comprehensive results
- **Adjust thresholds**: Lower values = more results, higher = more precise
- **Use --limit**: Control result count for large codebases

## Try It Yourself

Run these commands to see `ck` in action:

```bash
# Index the examples
ck --index examples/

# Compare different search modes on the same query
echo "=== Regex ==="
ck "error" examples/
echo "=== Semantic ==="  
ck --sem "error" examples/
echo "=== Hybrid ==="
ck --hybrid "error" examples/
```

Notice how each mode finds different but relevant results!

--- EXAMPLES.md ---
# ck (seek) - Usage Examples

This guide walks through practical examples of ck's capabilities, from basic grep replacement to advanced semantic search.

## Quick Setup

First, create a test project to explore ck's features:

```bash
mkdir ck-demo && cd ck-demo

# Create sample files
cat > auth.rs << EOF
use std::collections::HashMap;

pub struct AuthService {
    users: HashMap<String, User>,
}

impl AuthService {
    pub fn authenticate(&self, username: &str, password: &str) -> Result<Token, AuthError> {
        match self.users.get(username) {
            Some(user) if user.verify_password(password) => {
                Ok(Token::new(user.id))
            }
            _ => Err(AuthError::InvalidCredentials)
        }
    }

    pub fn handle_login_error(&self, error: AuthError) {
        log::error!("Authentication failed: {:?}", error);
    }
}
EOF

cat > database.py << EOF
import sqlite3
from typing import Optional

class DatabaseConnection:
    def __init__(self, db_path: str):
        self.connection = sqlite3.connect(db_path)
        
    def authenticate_user(self, username: str, password_hash: str) -> Optional[dict]:
        cursor = self.connection.cursor()
        try:
            cursor.execute(
                "SELECT id, username FROM users WHERE username = ? AND password_hash = ?",
                (username, password_hash)
            )
            return cursor.fetchone()
        except Exception as e:
            print(f"Database error: {e}")
            return None
            
    def handle_connection_error(self, error):
        log.error(f"Database connection failed: {error}")
EOF

cat > server.js << EOF
const express = require('express');
const bcrypt = require('bcrypt');

class AuthController {
    async login(req, res) {
        try {
            const { username, password } = req.body;
            const user = await this.findUser(username);
            
            if (!user || !bcrypt.compare(password, user.password)) {
                return res.status(401).json({ error: 'Authentication failed' });
            }
            
            const token = this.generateToken(user);
            res.json({ token });
        } catch (error) {
            this.handleError(error, res);
        }
    }
    
    handleError(error, res) {
        console.error('Login error:', error);
        res.status(500).json({ error: 'Internal server error' });
    }
}
EOF

cat > README.md << EOF
# Authentication Service

A multi-language authentication system with:

- Rust backend service
- Python database layer  
- JavaScript API endpoints

## Error Handling

All components implement proper error handling for authentication failures.
EOF
```

## 1. Basic Grep-Style Search (No Index Required)

ck works as a drop-in grep replacement:

```bash
# Find all mentions of "error"
ck "error" .

# Case-insensitive search for TODO items
ck -i "todo" .

# Show line numbers
ck -n "authentication" .

# Match whole words only
ck -w "error" .

# Fixed string search (no regex)
ck -F "AuthError::InvalidCredentials" .

# Show context around matches
ck -C 2 "authenticate" .
```

## 2. Create Search Index

Before using semantic or lexical search, create an index:

```bash
# Index current directory
ck index .

# Check what was indexed
ck status .

# Detailed index statistics
ck status . --verbose
```

## 3. Semantic Search - Find Similar Code

Semantic search understands meaning and finds conceptually related code:

```bash
# Find all authentication-related code
ck --sem "user authentication" .

# Find error handling patterns
ck --sem "error handling" .

# Find database operations
ck --sem "database connection" .

# Find login functionality
ck --sem "user login" .

# Limit to top 5 most relevant results
ck --sem "authentication" . --topk 5

# Filter by similarity score threshold (0.0-1.0)
ck --sem "authentication" . --threshold 0.7

# Show similarity scores in output
ck --sem "error handling" . --scores

# Combine threshold and score display
ck --sem "database" . --threshold 0.6 --scores

# Hybrid search with RRF threshold and scores
ck --hybrid "authentication" . --threshold 0.025 --scores
```

### What Makes This Powerful?

- Searching "error handling" finds `handle_login_error`, `handle_connection_error`, and `handleError`
- Searching "authentication" finds `authenticate`, `authenticate_user`, and `login` functions
- It understands that login, authentication, and user verification are related concepts

## 4. Lexical Search - BM25 Full-Text Ranking

Lexical search uses BM25 ranking for better phrase matching than regex:

```bash
# Full-text search with relevance ranking
ck --lex "authentication failed" .

# Better for multi-word phrases
ck --lex "database connection error" .

# Find documentation
ck --lex "error handling components" .
```

## 5. Hybrid Search - Best of Both Worlds

Combines regex pattern matching with semantic understanding:

```bash
# Find functions with "auth" that are semantically related to authentication
ck --hybrid "auth" .

# Pattern match + semantic relevance
ck --hybrid "error" --topk 10 .

# Filter hybrid results by RRF score threshold  
ck --hybrid "auth" --threshold 0.02 .

# Show scores for hybrid search (RRF scores ~0.01-0.05)
ck --hybrid "error" --scores .
```

## 6. JSON Output for Tools/Scripts

Get structured output for integration with other tools:

```bash
# JSON output with relevance scores
ck --json --sem "authentication" .

# Pipe to jq for processing
ck --json --sem "error" . | jq '.score'

# Get top 3 results as JSON
ck --json --topk 3 --lex "user login" .
```

## 7. Index Management

### Monitor Index Health

```bash
# Check index status
ck status .

# Detailed statistics
ck status . --verbose
```

### Update Index After Changes

```bash
# Edit a file
echo "// Added new auth method" >> auth.rs

# Index automatically updates on search, or force update:
ck index .

# Add single file to index
ck add auth.rs
```

### Clean Up Index

```bash
# Remove a file to create orphaned index entries
rm server.js

# Clean up orphaned entries only
ck clean . --orphans

# Remove entire index (will need to reindex for semantic search)
ck clean .
```

## 8. Advanced Examples

### Find All Error Handling Patterns

```bash
# Semantic search finds diverse error handling approaches
ck --sem "error handling" . --json | jq -r '.preview'
```

This finds:
- Rust: `Err(AuthError::InvalidCredentials)`
- Python: `except Exception as e:`
- JavaScript: `catch (error)`
- All `handle_*_error` functions

### Compare Search Methods

```bash
echo "Searching for 'auth' with different methods:"

echo "=== Regex (exact text matching) ==="
ck "auth" .

echo "=== Lexical (BM25 ranking) ==="  
ck --lex "auth" .

echo "=== Semantic (conceptual similarity) ==="
ck --sem "auth" .

echo "=== Hybrid (regex + semantic) ==="
ck --hybrid "auth" .
```

### Integration with Shell Scripts

```bash
#!/bin/bash
# Find security-related TODOs

echo "Security TODOs found:"
ck --sem "security vulnerability" . --json | \
    jq -r '"\(.file):\(.span.line_start): \(.preview)"'

echo -e "\nAuthentication issues:"
ck --hybrid "auth.*TODO" . --json | \
    jq -r '"\(.file): \(.preview)"'
```

## Performance and Use Cases

### When to Use Each Mode

- **Regex** (`ck "pattern"`): Exact text matching, grep replacement, no index needed
- **Lexical** (`--lex`): Multi-word phrases, document search, ranked results
- **Semantic** (`--sem`): Conceptual similarity, find related functionality, code exploration
- **Hybrid** (`--hybrid`): When you want both exact matches and similar concepts

### Index Management

- Index updates automatically during search (if >1 minute old)
- Use `ck status` to monitor index size and health
- Use `ck clean --orphans` after major file reorganization
- Use `ck --switch-model <model>` to rebuild the index with a different embedding model
- Add `--force` if you need to rebuild even when the model already matches
- Index persists in `.ck/` directory (add to `.gitignore`)

### File Type Support

ck indexes these file types:
- Code: `.rs`, `.py`, `.js`, `.ts`, `.go`, `.java`, `.c`, `.cpp`, `.rb`, `.php`, `.swift`, `.kt`
- Docs: `.md`, `.txt`, `.rst`  
- Config: `.json`, `.yaml`, `.toml`, `.xml`
- Scripts: `.sh`, `.bash`, `.ps1`, `.sql`

## Tips and Tricks

1. **Start with regex** for exact matching, then explore with semantic search
2. **Use `--topk`** to limit results when exploring large codebases  
3. **Use `--threshold`** to filter low-relevance results (semantic/lexical: 0.6-0.8, hybrid RRF: 0.02-0.05)  
4. **Use `--scores`** to see match quality and fine-tune your threshold
5. **Combine with shell tools**: `ck --json --sem "query" | jq`
6. **Index smaller directories** for faster semantic search
7. **Use `ck status -v`** to monitor index growth over time

## Troubleshooting

```bash
# No results from semantic search?
ck status .                    # Check if index exists
ck index . && ck --sem "query" # Create index first

# Index too large?
ck status . --verbose         # Check size statistics  
ck clean . --orphans          # Remove orphaned files

# Stale results?
ck index .                    # Force reindex
```

Happy seeking! 🔍


--- examples/code/api_client.js ---
/**
 * API Client for demonstrating ck semantic search capabilities
 * This file contains various patterns for HTTP requests, error handling, and authentication
 */

const https = require('https');
const crypto = require('crypto');

class ApiClient {
    constructor(baseUrl, options = {}) {
        this.baseUrl = baseUrl;
        this.timeout = options.timeout || 5000;
        this.retryCount = options.retryCount || 3;
        this.apiKey = options.apiKey;
        this.authToken = null;
    }

    /**
     * Authenticate with the API server
     * @param {string} username - User credentials
     * @param {string} password - User password
     * @returns {Promise<Object>} Authentication response
     */
    async authenticate(username, password) {
        try {
            const credentials = {
                username: username,
                password: password
            };

            const response = await this.post('/auth/login', credentials);
            
            if (response.token) {
                this.authToken = response.token;
                console.log('Authentication successful');
                return response;
            } else {
                throw new Error('No token received from authentication');
            }
        } catch (error) {
            console.error('Authentication failed:', error.message);
            throw new AuthenticationError('Failed to authenticate user', error);
        }
    }

    /**
     * Make HTTP GET request with error handling
     * @param {string} endpoint - API endpoint
     * @param {Object} params - Query parameters
     * @returns {Promise<Object>} Response data
     */
    async get(endpoint, params = {}) {
        const url = this.buildUrl(endpoint, params);
        
        return this.makeRequest('GET', url);
    }

    /**
     * Make HTTP POST request with retry logic
     * @param {string} endpoint - API endpoint
     * @param {Object} data - Request body
     * @returns {Promise<Object>} Response data
     */
    async post(endpoint, data) {
        const url = this.buildUrl(endpoint);
        
        let lastError;
        for (let attempt = 1; attempt <= this.retryCount; attempt++) {
            try {
                return await this.makeRequest('POST', url, data);
            } catch (error) {
                lastError = error;
                
                if (attempt < this.retryCount && this.shouldRetry(error)) {
                    console.warn(`Request failed (attempt ${attempt}), retrying...`);
                    await this.delay(1000 * attempt); // Exponential backoff
                    continue;
                }
                break;
            }
        }
        
        throw lastError;
    }

    /**
     * Upload file with progress tracking
     * @param {string} endpoint - Upload endpoint
     * @param {Buffer} fileData - File content
     * @param {string} filename - Original filename
     * @returns {Promise<Object>} Upload response
     */
    async uploadFile(endpoint, fileData, filename) {
        try {
            const formData = new FormData();
            formData.append('file', fileData, filename);
            
            const response = await this.makeRequest('POST', this.buildUrl(endpoint), formData, {
                'Content-Type': 'multipart/form-data'
            });
            
            console.log(`File ${filename} uploaded successfully`);
            return response;
        } catch (error) {
            console.error('File upload failed:', error);
            throw new FileUploadError(`Failed to upload ${filename}`, error);
        }
    }

    /**
     * Make the actual HTTP request
     * @private
     */
    async makeRequest(method, url, data = null, customHeaders = {}) {
        return new Promise((resolve, reject) => {
            const headers = {
                'Content-Type': 'application/json',
                'User-Agent': 'ck-demo-client/1.0',
                ...customHeaders
            };

            // Add authentication if available
            if (this.authToken) {
                headers.Authorization = `Bearer ${this.authToken}`;
            } else if (this.apiKey) {
                headers['X-API-Key'] = this.apiKey;
            }

            const requestOptions = {
                method: method,
                headers: headers,
                timeout: this.timeout
            };

            const request = https.request(url, requestOptions, (response) => {
                let responseData = '';
                
                response.on('data', (chunk) => {
                    responseData += chunk;
                });

                response.on('end', () => {
                    try {
                        if (response.statusCode >= 200 && response.statusCode < 300) {
                            const parsedData = JSON.parse(responseData);
                            resolve(parsedData);
                        } else {
                            reject(new HttpError(`HTTP ${response.statusCode}`, response.statusCode, responseData));
                        }
                    } catch (parseError) {
                        reject(new ParseError('Failed to parse response JSON', parseError));
                    }
                });
            });

            request.on('timeout', () => {
                request.destroy();
                reject(new TimeoutError('Request timeout'));
            });

            request.on('error', (error) => {
                reject(new NetworkError('Network request failed', error));
            });

            if (data && method !== 'GET') {
                const jsonData = typeof data === 'string' ? data : JSON.stringify(data);
                request.write(jsonData);
            }

            request.end();
        });
    }

    /**
     * Build complete URL with query parameters
     * @private
     */
    buildUrl(endpoint, params = {}) {
        const url = new URL(endpoint, this.baseUrl);
        
        Object.keys(params).forEach(key => {
            if (params[key] !== null && params[key] !== undefined) {
                url.searchParams.append(key, params[key]);
            }
        });
        
        return url.toString();
    }

    /**
     * Determine if request should be retried
     * @private
     */
    shouldRetry(error) {
        // Retry on network errors and 5xx server errors
        return error instanceof NetworkError || 
               error instanceof TimeoutError ||
               (error instanceof HttpError && error.statusCode >= 500);
    }

    /**
     * Delay helper for retry logic
     * @private
     */
    delay(ms) {
        return new Promise(resolve => setTimeout(resolve, ms));
    }

    /**
     * Generate secure hash for data integrity
     * @param {string} data - Data to hash
     * @returns {string} SHA-256 hash
     */
    generateHash(data) {
        try {
            return crypto.createHash('sha256').update(data).digest('hex');
        } catch (error) {
            throw new CryptographicError('Failed to generate hash', error);
        }
    }

    /**
     * Close client and cleanup resources
     */
    close() {
        this.authToken = null;
        console.log('API client closed');
    }
}

// Custom error classes for better error handling

class ApiClientError extends Error {
    constructor(message, cause = null) {
        super(message);
        this.name = this.constructor.name;
        this.cause = cause;
    }
}

class AuthenticationError extends ApiClientError {}
class NetworkError extends ApiClientError {}
class TimeoutError extends ApiClientError {}
class ParseError extends ApiClientError {}
class FileUploadError extends ApiClientError {}
class CryptographicError extends ApiClientError {}

class HttpError extends ApiClientError {
    constructor(message, statusCode, responseBody) {
        super(message);
        this.statusCode = statusCode;
        this.responseBody = responseBody;
    }
}

// Usage example
async function main() {
    const client = new ApiClient('https://api.example.com', {
        timeout: 10000,
        retryCount: 3,
        apiKey: 'your-api-key-here'
    });

    try {
        // Authenticate user
        await client.authenticate('user@example.com', 'secure-password');

        // Fetch user data
        const userData = await client.get('/users/profile');
        console.log('User data:', userData);

        // Update user settings
        const updateData = {
            name: 'John Doe',
            email: 'john@example.com',
            preferences: {
                theme: 'dark',
                notifications: true
            }
        };
        
        await client.post('/users/settings', updateData);
        console.log('Settings updated successfully');

    } catch (error) {
        if (error instanceof AuthenticationError) {
            console.error('Authentication failed. Please check your credentials.');
        } else if (error instanceof NetworkError) {
            console.error('Network error. Please check your connection.');
        } else if (error instanceof HttpError) {
            console.error(`Server error: ${error.statusCode} - ${error.message}`);
        } else {
            console.error('Unexpected error:', error.message);
        }
    } finally {
        client.close();
    }
}

// Export for use in other modules
module.exports = { ApiClient, AuthenticationError, NetworkError, HttpError };

// Run if called directly
if (require.main === module) {
    main().catch(console.error);
}

--- examples/code/full_section_demo.py ---
#!/usr/bin/env python3
"""
Full Section Demo - Python code examples for ck semantic search
This file demonstrates various Python patterns for testing ck's search capabilities
"""

import json
import logging
import asyncio
from typing import Dict, List, Optional, Union
from dataclasses import dataclass
from contextlib import asynccontextmanager


# Setup logging for error handling demonstration
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)


@dataclass
class UserProfile:
    """User profile data structure"""
    user_id: int
    username: str
    email: str
    is_active: bool = True
    metadata: Dict[str, Union[str, int]] = None

    def __post_init__(self):
        if self.metadata is None:
            self.metadata = {}


class DatabaseConnection:
    """Mock database connection for demonstration"""
    
    def __init__(self, connection_string: str):
        self.connection_string = connection_string
        self.is_connected = False
        self._users: Dict[int, UserProfile] = {}
    
    async def connect(self):
        """Establish database connection with error handling"""
        try:
            # Simulate connection attempt
            await asyncio.sleep(0.1)
            self.is_connected = True
            logger.info("Database connection established")
        except Exception as e:
            logger.error(f"Failed to connect to database: {e}")
            raise ConnectionError("Unable to establish database connection")
    
    async def get_user(self, user_id: int) -> Optional[UserProfile]:
        """Retrieve user profile with error handling"""
        if not self.is_connected:
            raise RuntimeError("Database not connected")
        
        try:
            return self._users.get(user_id)
        except KeyError:
            logger.warning(f"User {user_id} not found")
            return None
        except Exception as e:
            logger.error(f"Error retrieving user {user_id}: {e}")
            raise
    
    async def save_user(self, user: UserProfile):
        """Save user profile with validation and error handling"""
        if not self.is_connected:
            raise RuntimeError("Database not connected")
        
        # Data validation
        if not user.username or len(user.username) < 3:
            raise ValueError("Username must be at least 3 characters")
        
        if '@' not in user.email:
            raise ValueError("Invalid email format")
        
        try:
            self._users[user.user_id] = user
            logger.info(f"User {user.user_id} saved successfully")
        except Exception as e:
            logger.error(f"Failed to save user {user.user_id}: {e}")
            raise


class UserService:
    """User management service with comprehensive error handling"""
    
    def __init__(self, db: DatabaseConnection):
        self.db = db
        self._cache = {}
    
    async def authenticate_user(self, username: str, password: str) -> bool:
        """Authenticate user credentials"""
        if not username or not password:
            raise ValueError("Username and password are required")
        
        try:
            # In real implementation, check against database
            # This is a demo - never hardcode credentials!
            valid_users = {
                'admin': 'secret123',
                'user': 'password',
                'demo': 'demo123'
            }
            
            return valid_users.get(username) == password
        
        except Exception as e:
            logger.error(f"Authentication error for user {username}: {e}")
            return False
    
    async def create_user(self, username: str, email: str) -> UserProfile:
        """Create new user with validation and error handling"""
        try:
            # Validate input
            if not username or len(username) < 3:
                raise ValueError("Username must be at least 3 characters")
            
            if not email or '@' not in email:
                raise ValueError("Valid email address required")
            
            # Create user profile
            user_id = hash(username) % 10000  # Simple ID generation for demo
            user = UserProfile(
                user_id=user_id,
                username=username,
                email=email,
                metadata={'created_by': 'system', 'version': 1}
            )
            
            # Save to database
            await self.db.save_user(user)
            
            logger.info(f"Created user: {username} ({user_id})")
            return user
            
        except ValueError as e:
            logger.warning(f"Validation error creating user {username}: {e}")
            raise
        except Exception as e:
            logger.error(f"Unexpected error creating user {username}: {e}")
            raise RuntimeError("User creation failed")
    
    async def process_user_data(self, user_id: int) -> Dict:
        """Process user data with comprehensive error handling"""
        try:
            user = await self.db.get_user(user_id)
            if not user:
                raise ValueError(f"User {user_id} not found")
            
            # Simulate data processing
            processed_data = {
                'user_info': {
                    'id': user.user_id,
                    'name': user.username,
                    'contact': user.email,
                    'status': 'active' if user.is_active else 'inactive'
                },
                'metadata': user.metadata,
                'processed_at': asyncio.get_event_loop().time()
            }
            
            return processed_data
            
        except ValueError as e:
            logger.warning(f"Data processing validation error: {e}")
            raise
        except Exception as e:
            logger.error(f"Error processing user data for {user_id}: {e}")
            raise RuntimeError("Data processing failed")


@asynccontextmanager
async def database_session(connection_string: str):
    """Context manager for database sessions with proper cleanup"""
    db = DatabaseConnection(connection_string)
    try:
        await db.connect()
        yield db
    except Exception as e:
        logger.error(f"Database session error: {e}")
        raise
    finally:
        # Cleanup would go here in real implementation
        logger.info("Database session closed")


async def main():
    """Main function demonstrating error handling patterns"""
    connection_string = "postgresql://localhost/demo"
    
    try:
        async with database_session(connection_string) as db:
            service = UserService(db)
            
            # Test user creation
            try:
                user = await service.create_user("demo_user", "demo@example.com")
                logger.info(f"Created user: {user}")
                
                # Test authentication
                is_authenticated = await service.authenticate_user("demo_user", "wrong_password")
                logger.info(f"Authentication result: {is_authenticated}")
                
                # Test data processing
                processed = await service.process_user_data(user.user_id)
                logger.info(f"Processed data: {json.dumps(processed, indent=2)}")
                
            except ValueError as e:
                logger.warning(f"Validation error: {e}")
            except RuntimeError as e:
                logger.error(f"Runtime error: {e}")
                
    except ConnectionError as e:
        logger.error(f"Database connection failed: {e}")
        return 1
    except Exception as e:
        logger.error(f"Unexpected error: {e}")
        return 1
    
    return 0


if __name__ == "__main__":
    try:
        exit_code = asyncio.run(main())
        exit(exit_code)
    except KeyboardInterrupt:
        logger.info("Application interrupted by user")
        exit(130)
    except Exception as e:
        logger.error(f"Fatal error: {e}")
        exit(1)

--- examples/code/large_function.py ---
def large_complex_function(data, config, options):
    """
    This is a large function that might exceed token limits.
    It processes data according to configuration and options.
    """
    # Initialize variables
    result = []
    processed_count = 0
    error_count = 0
    
    # Configuration validation
    if not config:
        raise ValueError("Configuration cannot be empty")
    
    if not isinstance(config, dict):
        raise TypeError("Configuration must be a dictionary")
    
    # Check required configuration keys
    required_keys = ['api_endpoint', 'timeout', 'retry_count', 'batch_size']
    for key in required_keys:
        if key not in config:
            raise KeyError(f"Required configuration key '{key}' is missing")
    
    # Validate configuration values
    if config['timeout'] <= 0:
        raise ValueError("Timeout must be positive")
    
    if config['retry_count'] < 0:
        raise ValueError("Retry count cannot be negative")
    
    if config['batch_size'] <= 0:
        raise ValueError("Batch size must be positive")
    
    # Process options
    debug_mode = options.get('debug', False)
    verbose = options.get('verbose', False)
    max_workers = options.get('max_workers', 4)
    
    # Input data validation
    if not data:
        if debug_mode:
            print("Warning: No data provided for processing")
        return []
    
    if not isinstance(data, (list, tuple)):
        data = [data]
    
    # Process data in batches
    batch_size = config['batch_size']
    total_batches = len(data) // batch_size + (1 if len(data) % batch_size else 0)
    
    for batch_index in range(total_batches):
        start_idx = batch_index * batch_size
        end_idx = min(start_idx + batch_size, len(data))
        batch = data[start_idx:end_idx]
        
        if verbose:
            print(f"Processing batch {batch_index + 1}/{total_batches} ({len(batch)} items)")
        
        # Process each item in the batch
        batch_results = []
        for item_index, item in enumerate(batch):
            try:
                # Validate item structure
                if not isinstance(item, dict):
                    raise TypeError(f"Item at index {start_idx + item_index} must be a dictionary")
                
                # Check required fields
                required_fields = ['id', 'type', 'data']
                for field in required_fields:
                    if field not in item:
                        raise KeyError(f"Required field '{field}' missing in item {start_idx + item_index}")
                
                # Process based on type
                processed_item = None
                if item['type'] == 'string':
                    processed_item = process_string_item(item, config, options)
                elif item['type'] == 'number':
                    processed_item = process_number_item(item, config, options)
                elif item['type'] == 'array':
                    processed_item = process_array_item(item, config, options)
                elif item['type'] == 'object':
                    processed_item = process_object_item(item, config, options)
                else:
                    raise ValueError(f"Unknown item type: {item['type']}")
                
                # Apply transformations
                if 'transformations' in options:
                    for transform in options['transformations']:
                        processed_item = apply_transformation(processed_item, transform)
                
                # Add to batch results
                batch_results.append(processed_item)
                processed_count += 1
                
                if debug_mode:
                    print(f"Successfully processed item {start_idx + item_index}")
                
            except Exception as e:
                error_count += 1
                error_msg = f"Error processing item {start_idx + item_index}: {str(e)}"
                
                if debug_mode:
                    print(error_msg)
                
                # Handle error based on options
                if options.get('ignore_errors', False):
                    continue
                elif options.get('collect_errors', False):
                    batch_results.append({'error': error_msg, 'original_item': item})
                else:
                    raise RuntimeError(error_msg) from e
        
        # Add batch results to overall results
        result.extend(batch_results)
        
        # Progress callback if provided
        if 'progress_callback' in options:
            progress = (batch_index + 1) / total_batches
            options['progress_callback'](progress, processed_count, error_count)
    
    # Post-processing
    if 'post_processors' in options:
        for processor in options['post_processors']:
            result = processor(result, config, options)
    
    # Final validation
    if 'validators' in options:
        for validator in options['validators']:
            if not validator(result, config, options):
                raise ValueError("Result validation failed")
    
    # Logging and statistics
    if verbose:
        print(f"Processing completed:")
        print(f"  Total items processed: {processed_count}")
        print(f"  Total errors: {error_count}")
        print(f"  Success rate: {processed_count / len(data) * 100:.2f}%")
    
    return result

def process_string_item(item, config, options):
    """Process string-type items with various transformations."""
    data = item['data']
    
    # Basic validation
    if not isinstance(data, str):
        raise TypeError("String item data must be a string")
    
    # Apply string transformations
    if options.get('trim_whitespace', True):
        data = data.strip()
    
    if options.get('lowercase', False):
        data = data.lower()
    
    if options.get('uppercase', False):
        data = data.upper()
    
    # Length validation
    min_length = options.get('min_string_length', 0)
    max_length = options.get('max_string_length', float('inf'))
    
    if len(data) < min_length:
        raise ValueError(f"String length {len(data)} below minimum {min_length}")
    
    if len(data) > max_length:
        raise ValueError(f"String length {len(data)} exceeds maximum {max_length}")
    
    return {
        'id': item['id'],
        'type': 'string',
        'processed_data': data,
        'original_length': len(item['data']),
        'processed_length': len(data)
    }

def process_number_item(item, config, options):
    """Process number-type items with validation and transformations."""
    data = item['data']
    
    # Basic validation
    if not isinstance(data, (int, float)):
        raise TypeError("Number item data must be numeric")
    
    # Range validation
    min_value = options.get('min_number_value', float('-inf'))
    max_value = options.get('max_number_value', float('inf'))
    
    if data < min_value:
        raise ValueError(f"Number {data} below minimum {min_value}")
    
    if data > max_value:
        raise ValueError(f"Number {data} exceeds maximum {max_value}")
    
    # Transformations
    processed_data = data
    
    if options.get('round_numbers', False):
        decimals = options.get('round_decimals', 0)
        processed_data = round(processed_data, decimals)
    
    if options.get('abs_numbers', False):
        processed_data = abs(processed_data)
    
    return {
        'id': item['id'],
        'type': 'number',
        'processed_data': processed_data,
        'original_value': data,
        'is_modified': processed_data != data
    }

def process_array_item(item, config, options):
    """Process array-type items with element validation."""
    data = item['data']
    
    # Basic validation
    if not isinstance(data, (list, tuple)):
        raise TypeError("Array item data must be a list or tuple")
    
    # Length validation
    min_length = options.get('min_array_length', 0)
    max_length = options.get('max_array_length', float('inf'))
    
    if len(data) < min_length:
        raise ValueError(f"Array length {len(data)} below minimum {min_length}")
    
    if len(data) > max_length:
        raise ValueError(f"Array length {len(data)} exceeds maximum {max_length}")
    
    # Process each element
    processed_elements = []
    for i, element in enumerate(data):
        try:
            # Element validation based on options
            if 'array_element_type' in options:
                expected_type = options['array_element_type']
                if expected_type == 'string' and not isinstance(element, str):
                    raise TypeError(f"Array element {i} must be string")
                elif expected_type == 'number' and not isinstance(element, (int, float)):
                    raise TypeError(f"Array element {i} must be number")
            
            processed_elements.append(element)
        except Exception as e:
            if options.get('skip_invalid_array_elements', False):
                continue
            else:
                raise RuntimeError(f"Error processing array element {i}: {str(e)}") from e
    
    return {
        'id': item['id'],
        'type': 'array',
        'processed_data': processed_elements,
        'original_length': len(data),
        'processed_length': len(processed_elements)
    }

def process_object_item(item, config, options):
    """Process object-type items with nested structure validation."""
    data = item['data']
    
    # Basic validation
    if not isinstance(data, dict):
        raise TypeError("Object item data must be a dictionary")
    
    # Required fields validation
    if 'required_object_fields' in options:
        for field in options['required_object_fields']:
            if field not in data:
                raise KeyError(f"Required object field '{field}' is missing")
    
    # Process nested data
    processed_data = {}
    for key, value in data.items():
        # Key validation
        if options.get('validate_object_keys', False):
            if not isinstance(key, str):
                raise TypeError(f"Object key '{key}' must be a string")
            
            if len(key) == 0:
                raise ValueError("Object keys cannot be empty")
        
        # Value processing based on type
        if isinstance(value, str) and options.get('process_nested_strings', False):
            processed_value = value.strip() if options.get('trim_nested_whitespace', True) else value
        elif isinstance(value, (int, float)) and options.get('process_nested_numbers', False):
            processed_value = round(value, 2) if options.get('round_nested_numbers', False) else value
        elif isinstance(value, list) and options.get('process_nested_arrays', False):
            processed_value = [v for v in value if v is not None] if options.get('filter_null_values', False) else value
        else:
            processed_value = value
        
        processed_data[key] = processed_value
    
    return {
        'id': item['id'],
        'type': 'object',
        'processed_data': processed_data,
        'field_count': len(processed_data)
    }

def apply_transformation(item, transform):
    """Apply a transformation function to an item."""
    if callable(transform):
        return transform(item)
    elif isinstance(transform, dict):
        # Dictionary-based transformation
        if 'type' in transform:
            if transform['type'] == 'rename_field':
                old_name = transform['old_name']
                new_name = transform['new_name']
                if old_name in item and old_name != new_name:
                    item[new_name] = item.pop(old_name)
            elif transform['type'] == 'add_field':
                field_name = transform['field_name']
                field_value = transform['field_value']
                item[field_name] = field_value
    
    return item

--- examples/text_samples/wiki_articles/Algorithm.txt ---
# Algorithm
Humpty dumpty
In mathematics and computer science, an algorithm ( ) is a finite sequence of mathematically rigorous instructions, typically used to solve a class of specific problems or to perform a computation. Algorithms are used as specifications for performing calculations and data processing. More advanced algorithms can use conditionals to divert the code execution through various routes (referred to as automated decision-making) and deduce valid inferences (referred to as automated reasoning). In contrast, a heuristic is an approach to solving problems without well-defined correct or optimal results. For example, although social media recommender systems are commonly called "algorithms", they actually rely on heuristics as there is no truly "correct" recommendation. As an effective method, an algorithm can be expressed within a finite amount of space and time and in a well-defined formal language for calculating a function. Starting from an initial state and initial input (perhaps empty), the instructions describe a computation that, when executed, proceeds through a finite number of well-defined successive states, eventually producing "output" and terminating at a final ending state. The transition from one state to the next is not necessarily deterministic; some algorithms, known as randomized algorithms, incorporate random input.

--- examples/text_samples/wiki_articles/Art.txt ---
# Art

Art is a diverse range of cultural activity centered around works utilizing creative or imaginative talents, which are expected to evoke a worthwhile experience, generally through an expression of emotional power, conceptual ideas, technical proficiency, or beauty. There is no generally agreed definition of what constitutes art, and its interpretation has varied greatly throughout history and across cultures. In the Western tradition, the three classical branches of visual art are painting, sculpture, and architecture. Theatre, dance, and other performing arts, as well as literature, music, film and other media such as interactive media, are included in a broader definition of "the arts". Until the 17th century, art referred to any skill or mastery and was not differentiated from crafts or sciences. In modern usage after the 17th century, where aesthetic considerations are paramount, the fine arts are separated and distinguished from acquired skills in general, such as the decorative or applied arts. The nature of art and related concepts, such as creativity and interpretation, are explored in a branch of philosophy known as aesthetics. The resulting artworks are studied in the professional fields of art criticism and the history of art.

--- examples/text_samples/wiki_articles/Artificial_intelligence.txt ---
# Artificial intelligence

Artificial intelligence (AI) is the capability of computational systems to perform tasks typically associated with human intelligence, such as learning, reasoning, problem-solving, perception, and decision-making. It is a field of research in computer science that develops and studies methods and software that enable machines to perceive their environment and use learning and intelligence to take actions that maximize their chances of achieving defined goals. High-profile applications of AI include advanced web search engines (e.g., Google Search); recommendation systems (used by YouTube, Amazon, and Netflix); virtual assistants (e.g., Google Assistant, Siri, and Alexa); autonomous vehicles (e.g., Waymo); generative and creative tools (e.g., language models and AI art); and superhuman play and analysis in strategy games (e.g., chess and Go). However, many AI applications are not perceived as AI: "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." Various subfields of AI research are centered around particular goals and the use of particular tools. The traditional goals of AI research include learning, reasoning, knowledge representation, planning, natural language processing, perception, and support for robotics. To reach these goals, AI researchers have adapted and integrated a wide range of techniques, including search and mathematical optimization, formal logic, artificial neural networks, and methods based on statistics, operations research, and economics. AI also draws upon psychology, linguistics, philosophy, neuroscience, and other fields. Some companies, such as OpenAI, Google DeepMind and Meta, aim to create artificial general intelligence (AGI)—AI that can complete virtually any cognitive task at least as well as a human. Artificial intelligence was founded as an academic discipline in 1956, and the field went through multiple cycles of optimism throughout its history, followed by periods of disappointment and loss of funding, known as AI winters. Funding and interest vastly increased after 2012 when graphics processing units started being used to accelerate neural networks and deep learning outperformed previous AI techniques. This growth accelerated further after 2017 with the transformer architecture. In the 2020s, an ongoing period of rapid progress in advanced generative AI became known as the AI boom. Generative AI's ability to create and modify content has led to several unintended consequences and harms, which has raised ethical concerns about AI's long-term effects and potential existential risks, prompting discussions about regulatory policies to ensure the safety and benefits of the technology.

--- examples/text_samples/wiki_articles/Biology.txt ---
# Biology

Biology is the scientific study of life and living organisms. It is a broad natural science that encompasses a wide range of fields and unifying principles that explain the structure, function, growth, origin, evolution, and distribution of life. Central to biology are five fundamental themes: the cell as the basic unit of life, genes and heredity as the basis of inheritance, evolution as the driver of biological diversity, energy transformation for sustaining life processes, and the maintenance of internal stability (homeostasis). Biology examines life across multiple levels of organization, from molecules and cells to organisms, populations, and ecosystems. Subdisciplines include molecular biology, physiology, ecology, evolutionary biology, developmental biology, and systematics, among others. Each of these fields applies a range of methods to investigate biological phenomena, including observation, experimentation, and mathematical modeling. Modern biology is grounded in the theory of evolution by natural selection, first articulated by Charles Darwin, and in the molecular understanding of genes encoded in DNA. The discovery of the structure of DNA and advances in molecular genetics have transformed many areas of biology, leading to applications in medicine, agriculture, biotechnology, and environmental science. Life on Earth is believed to have originated over 3.7 billion years ago. Today, it includes a vast diversity of organisms—from single-celled archaea and bacteria to complex multicellular plants, fungi, and animals. Biologists classify organisms based on shared characteristics and evolutionary relationships, using taxonomic and phylogenetic frameworks. These organisms interact with each other and with their environments in ecosystems, where they play roles in energy flow and nutrient cycling. As a constantly evolving field, biology incorporates new discoveries and technologies that enhance the understanding of life and its processes, while contributing to solutions for challenges such as disease, climate change, and biodiversity loss.

--- examples/text_samples/wiki_articles/Chemistry.txt ---
# Chemistry

Chemistry is the scientific study of the properties and behavior of matter. It is a physical science within the natural sciences that studies the chemical elements that make up matter and compounds made of atoms, molecules and ions: their composition, structure, properties, behavior and the changes they undergo during reactions with other substances. Chemistry also addresses the nature of chemical bonds in chemical compounds. In the scope of its subject, chemistry occupies an intermediate position between physics and biology. It is sometimes called the central science because it provides a foundation for understanding both basic and applied scientific disciplines at a fundamental level. For example, chemistry explains aspects of plant growth (botany), the formation of igneous rocks (geology), how atmospheric ozone is formed and how environmental pollutants are degraded (ecology), the properties of the soil on the Moon (cosmochemistry), how medications work (pharmacology), and how to collect DNA evidence at a crime scene (forensics). Chemistry has existed under various names since ancient times. It has evolved, and now chemistry encompasses various areas of specialisation, or subdisciplines, that continue to increase in number and interrelate to create further interdisciplinary fields of study. The applications of various fields of chemistry are used frequently for economic purposes in the chemical industry.

--- CHANGELOG.md ---
# Changelog

All notable changes to this project will be documented in this file.

## [Unreleased]

### Added
- **VitePress documentation site**: Comprehensive documentation with improved navigation, search, and structure in `docs-site/` directory
- **Documentation features**: Guide pages, feature documentation, CLI reference, embedding model guide, architecture docs, and contributing guides
- **Local search**: Built-in search functionality in documentation site

### Technical
- **Self-contained docs**: All documentation tooling isolated in docs-site/ with independent build process using pnpm and VitePress
- **Node.js integration**: Documentation site uses Node.js 18+, pnpm 10+, and VitePress 1.6+ for modern documentation experience
- **GitHub integration**: Edit links and social links configured for easy contribution

## [0.7.1] - 2025-11-05

### Fixed
- **Hierarchical .ckignore support**: Fixed MCP daemon subdirectory search failures and indexing performance issues by implementing proper hierarchical .ckignore file loading using WalkBuilder's custom ignore filename support (PR #84)
- **Subdirectory search**: MCP searches in subdirectories now correctly respect parent directories' .ckignore files, matching .gitignore behavior
- **Indexing performance**: Eliminated repeated indexing of ignored files, significantly improving indexing speed in large repositories

### Technical
- **FileCollectionOptions refactor**: Replaced parameter threading anti-pattern with unified config struct for cleaner architecture
- **WalkBuilder integration**: Configured with `.add_custom_ignore_filename(".ckignore")` for hierarchical ignore file support
- **CLI flag addition**: Added `--no-ckignore` flag to disable .ckignore support when needed
- **Test coverage**: Added regression test `test_subdirectory_search_uses_parent_ckignore` to prevent future regressions
- **Dependencies**: Bumped vite from 5.4.20 to 5.4.21 in docs-site (PR #82)

## [0.7.0] - 2025-10-13

### Added
- **Chunk-level incremental indexing**: Smart caching system that reuses embeddings for unchanged chunks, dramatically improving reindexing performance (80-90% cache hit rate for typical code changes)
- **Content-aware cache invalidation**: Hash-based invalidation using blake3(chunk_text + leading_trivia + trailing_trivia) ensures doc comment and whitespace changes properly invalidate cache
- **Model compatibility enforcement**: Prevents silent embedding corruption by validating model consistency across indexing operations with clear error messages and recovery guidance
- **Chunk hash versioning**: Manifest tracking of hash scheme version (v2) for future compatibility and reliable version detection

### Performance
- **Selective re-embedding**: Only changed chunks are re-embedded; unchanged chunks reuse cached embeddings from previous index
- **Cache hit validation**: Both hash match AND dimension match required before reusing cached embeddings
- **Typical performance**: For code changes affecting 10-20% of file chunks, 80-90% of embeddings are reused from cache

### Technical
- **Blake3 hashing**: Fast cryptographic hashing of chunk content including all trivia for reliable change detection
- **Sidecar-based cache**: Old sidecars loaded into memory to build chunk_hash → embedding cache for efficient lookup
- **Model validation**: Index operations validate embedding model matches existing index and return actionable errors on mismatch
- **Backward compatibility**: Existing manifests auto-upgrade to chunk_hash_version v2 on load
- **Comprehensive testing**: All 181 tests passing with full coverage of cache invalidation, model validation, and version tracking

## [0.6.0] - 2025-10-12

### Added
- **MCP Server**: Full Model Context Protocol implementation with stdio transport for AI agent integration
- **Pagination Support**: Cursor-based pagination for all search modes (page_size: 1-200, default: 50)
- **Session Management**: TTL-based session cleanup for paginated results with automatic expiration (60s)
- **MCP Search Tools**: `semantic_search`, `regex_search`, `hybrid_search`, `lexical_search` with unified interface
- **MCP Index Tools**: `index_status`, `reindex`, `health_check` for index management
- **CLI Heatmap Visualization**: Color-coded similarity scores with RGB gradient highlighting (red→yellow→green)
- **Enhanced Visual Output**: Unicode box drawing and sophisticated match highlighting in CLI
- **Near-Miss Tracking**: Track closest result below threshold for better search feedback
- **Fallback Strategies**: Automatic fallback from semantic to lexical search when embeddings unavailable
- **Graceful Error Handling**: Skip stale index entries when files no longer exist

### Fixed
- **Mixed Line Endings**: Proper handling of Unix (\n), Windows (\r\n), and Mac (\r) line endings
- **Span Validation**: Prevent invalid spans with zero line numbers using `Span::new()` validation
- **Streaming File Operations**: Memory-efficient line extraction without loading entire files
- **PDF Content Resolution**: Better content path resolution and caching for PDF files

### Technical
- **MCP Protocol**: Full implementation with tool discovery, validation, and error handling
- **Session Cleanup**: LRU-based eviction with periodic cleanup task (every 30s)
- **Streaming Reads**: Optimized `extract_lines_from_file()` for minimal memory footprint
- **SearchResults Enhancement**: Added `closest_below_threshold` field for improved UX
- **Comprehensive Testing**: 7 new MCP integration tests covering pagination, validation, and edge cases
- **Line Ending Support**: `split_lines_with_endings()` tracks exact byte lengths per line
- **TUI Refactoring**: Extracted TUI functionality into dedicated `ck-tui` crate (3,084 lines)
- **Modular Architecture**: Clean separation of TUI components with public API
- **Config Persistence**: TUI preferences saved to `~/.config/ck/tui.json`

### Breaking Changes
- **Span Construction**: Use `Span::new()` for validated construction instead of struct literals (backward compatible via `Span::new_unchecked()`)

## [0.5.3] - 2025-09-29

### Added
- **`.ckignore` file support**: Automatic creation of `.ckignore` file with sensible defaults for persistent exclusion patterns
- **Media file exclusions**: Images (png, jpg, gif, svg, etc.), videos (mp4, avi, mov, etc.), and audio files (mp3, wav, flac, etc.) excluded by default
- **Config file exclusions**: JSON and YAML files excluded from indexing by default to reduce noise in search results
- **`--no-ckignore` flag**: Option to bypass `.ckignore` patterns when needed
- **Persistent patterns**: Exclusion patterns persist across searches without needing command-line flags each time

### Fixed
- **Exclusion pattern persistence** (issue #67): Patterns now persist in `.ckignore` instead of requiring `--exclude` flags on every search
- **Media file indexing** (issue #66): Images, videos, and other binary files no longer indexed by default
- **Config file noise** (issue #27): JSON/YAML config files excluded to focus search on actual code

### Technical
- **Additive pattern merging**: `.gitignore` + `.ckignore` + CLI + defaults all merge together (not mutually exclusive)
- **Auto-creation on first index**: `.ckignore` created automatically at repository root during first indexing
- **Glob pattern syntax**: Uses same pattern syntax as `.gitignore` for familiarity
- **Comprehensive test coverage**: 4 new tests covering creation, parsing, and exclusion logic

## [0.4.7] - 2025-09-19

### Added
- **Model switching command**: New `--switch-model` flag for seamless embedding model transitions with intelligent rebuild detection
- **Force rebuild option**: `--force` flag for explicit index rebuilding when switching models
- **Model resolution system**: Smart model management that respects existing index configurations and provides clear conflict guidance
- **Enhanced status display**: Index status now shows which embedding model and dimensions are in use
- **Search model validation**: Prevents mixing embedding models during search operations with actionable error messages

### Fixed
- **Windows atomic writes**: Fixed critical Windows compatibility issue where index files could become corrupted during writes
- **Embedding dimension mismatches**: Comprehensive validation preventing crashes from mixed embedding models with clear user guidance
- **Model consistency**: Enforced consistent embedding model usage across index lifecycle (build, search, update)
- **Clippy compliance**: Resolved all compiler warnings to meet strict CI requirements

### Technical
- **Atomic file operations**: Uses `tempfile::NamedTempFile` for cross-platform atomic writes with proper sync guarantees
- **Model registry integration**: Centralized model management with alias support and dimension tracking
- **Enhanced error messages**: User-friendly error messages with exact commands to resolve issues (e.g., "run `ck --clean .` then rebuild")
- **Legacy code cleanup**: Removed 338 lines of unused ANN semantic search implementation
- **Interrupt handling**: Proper Ctrl+C handling during indexing with graceful cleanup

## [0.4.5] - 2025-09-13

### Added
- **Enhanced token-based chunking**: Implemented model-specific token-aware chunking using HuggingFace tokenizers for precise token counting instead of character estimation
- **Model-specific configurations**: Chunks now sized according to model capacity - 1024 tokens for large models (nomic/jina) vs 400 tokens for small models (bge-small)
- **Streamlined --inspect command**: Enhanced file inspection showing token counts per chunk, language detection, and clean visualization without visual noise
- **FastEmbed capacity utilization**: Configured FastEmbed to use full model capacity (8192 tokens for nomic/jina models vs previous 512 token truncation)
- **Indexing progress transparency**: Added model name and chunk configuration display during indexing operations

### Fixed
- **Token estimation accuracy**: Replaced rough character-based estimation with actual model tokenizers for precise chunking
- **Model capacity underutilization**: Fixed FastEmbed configuration to use full 8K context for large models instead of 512-token default
- **Clippy compliance**: Resolved all compiler warnings to meet CI/CD standards with `-D warnings` flag
- **Unused code cleanup**: Removed dead code and properly annotated intentional allowances for CI compliance

### Technical
- **HuggingFace tokenizer integration**: Added hf-hub and tokenizers dependencies for precise token counting
- **Model-aware chunking system**: `get_model_chunk_config()` function providing balanced precision vs context chunking strategy
- **Enhanced --inspect visualization**: Complete rewrite showing essential chunking information without progress bar clutter
- **Comprehensive quality checks**: All 88 tests passing with clippy compliance and code formatting standards

## [0.4.4] - 2025-09-13

### Fixed
- **`--add` command argument parsing**: Fixed issue where file paths were incorrectly parsed as pattern arguments, preventing single file additions to the index
- **Empty pattern behavior**: Empty regex patterns now match each line once (consistent with grep/ripgrep) instead of matching at every character position causing massive duplication

## [0.4.3] - 2025-09-13

### Added
- **Enhanced embedding models**: Added support for Nomic V1.5 (8192 tokens, 768 dimensions) and Jina Code (8192 tokens, code-specialized) models
- **Model selection**: New `--model` flag for choosing embedding model during indexing (`bge-small`, `nomic-v1.5`, `jina-code`)
- **Index-time model configuration**: Model selection is now properly configured at index creation time and stored in index manifest
- **Automatic model detection**: Search operations automatically use the model stored in the index manifest
- **Reranking support**: Added cross-encoder reranking with `--rerank` flag and `--rerank-model` option for improved search relevance
- **Striding for large chunks**: Implemented text striding with overlap for chunks exceeding model token limits
- **Token estimation**: Added token counting utilities to optimize chunk sizes for different models

### Fixed
- **Ctrl-C interrupt handling**: Fixed issue where indexing could not be properly cancelled - now uses `try_for_each` to stop all parallel workers immediately
- **Model compatibility checking**: Index operations now validate model compatibility and provide clear error messages for mismatches

### Technical
- **Model registry system**: New `ck-models` crate with centralized model configuration and limits
- **Index manifest enhancement**: Added `embedding_model` and `embedding_dimensions` fields to track model used for indexing
- **Backward compatibility**: Existing indexes without model metadata continue to work with default BGE model
- **Architecture fix**: Corrected design where model selection was incorrectly a search-time option instead of index-time configuration

### Documentation
- **README model guide**: Added comprehensive section explaining embedding model options and their trade-offs
- **CLI help improvements**: Enhanced help text with clear model selection examples and implications

## [0.4.2] - 2025-09-11

### Fixed
- **Hidden file indexing bug**: Fixed critical bug where hidden directories (especially `.git`) were being indexed despite exclusion patterns
- **Semantic search pollution**: Eliminated `.git` files appearing in semantic search results for unrelated queries
- **Index size reduction**: Significantly reduced index size by properly excluding hidden files and directories

### Technical
- **WalkBuilder configuration**: Changed `.hidden(false)` to `.hidden(true)` to respect hidden file conventions
- **Exclusion pattern enforcement**: Hidden file exclusion now takes precedence, preventing override patterns from being ignored
- **Performance improvement**: Reduced indexing time and storage by not processing `.git` and other hidden directories

## [0.4.1] - 2025-09-10

### Added
- **JSONL output format**: Stream-friendly `--jsonl` flag for AI agent workflows with structured output
- **No-snippet mode**: `--no-snippet` flag for metadata-only output to reduce bandwidth for agents
- **Agent documentation**: Comprehensive README section explaining JSONL benefits over traditional JSON
- **Agent examples**: Python code demonstrating stream processing patterns for AI workflows
- **UTF-8 warning suppression**: Eliminated noisy warnings for binary files in .git directories
- **JSONL output format**: Stream-friendly `--jsonl` flag for AI agent workflows with structured output
- **No-snippet mode**: `--no-snippet` flag for metadata-only output to reduce bandwidth for agents
- **Agent documentation**: Comprehensive README section explaining JSONL benefits over traditional JSON
- **Agent examples**: Python code demonstrating stream processing patterns for AI workflows
- **UTF-8 warning suppression**: Eliminated noisy warnings for binary files in .git directories

### Technical
- **JsonlSearchResult struct**: New agent-friendly output format with conversion methods
- **Extended SearchResult**: Added chunk_hash and index_epoch fields for future agent features
- **Comprehensive test coverage**: 4 new integration tests validating JSONL functionality
- **Updated help text**: Dedicated JSONL section explaining streaming benefits for agents
- **Phase 1 PRD**: Complete specification for agent-ready code navigation features

### Why JSONL for AI Agents?
- **Streaming friendly**: Process results as they arrive, no waiting for complete response
- **Memory efficient**: Parse one result at a time, not entire array into memory
- **Error resilient**: Malformed lines don't break entire response
- **Standard format**: Used by OpenAI, Anthropic, and modern ML pipelines

## [0.3.9] - 2025-09-10

### Added
- **Streaming producer-consumer indexing**: Implemented efficient streaming architecture for large-scale indexing operations
- **Memory-efficient processing**: Reduces memory footprint during indexing of large codebases
- **Performance optimization**: Better resource utilization through streaming data flow

### Technical
- **Producer-consumer pattern**: Separates file discovery from processing for better parallelization
- **Streaming integration**: Compatible with existing smart update and exclude pattern functionality

## [0.3.8] - 2025-09-09

### Added
- **Enhanced model caching documentation**: Updated README with comprehensive information about embedding model cache locations
- **Platform-specific cache paths**: Documented cache directories for Linux/macOS (`~/.cache/ck/models/`), Windows (`%LOCALAPPDATA%\ck\cache\models\`), and fallback locations
- **Model download transparency**: Clear documentation of where fastembed stores ONNX models when downloaded during indexing

### Fixed
- **Documentation accuracy**: Removed outdated `.fastembed_cache` references and provided correct cache path information
- **FAQ section**: Added frequently asked questions about embedding model storage and management

## [0.3.7] - 2025-09-08

### Improved
- **Smart binary detection**: Replaced restrictive extension-based file detection with ripgrep-style content analysis using NUL byte detection
- **Broader text file support**: Now automatically indexes log files (`.log`), config files (`.env`, `.conf`), and any other text format regardless of extension
- **Improved accuracy**: Files without extensions containing text content are now correctly detected and indexed
- **Binary file exclusion**: Files containing NUL bytes (executables, images, etc.) are correctly identified as binary and excluded from indexing
- **Performance**: Fast detection using only the first 8KB of file content, similar to ripgrep's approach

### Technical
- **Content-based detection**: `is_text_file()` function now reads file content instead of checking against a hardcoded extension allowlist
- **Test coverage**: Added comprehensive tests for binary detection with various file types and edge cases

## [0.3.6] - 2025-09-08

### Fixed
- **Exclude patterns functionality**: Fixed critical bug where `--exclude` patterns were completely ignored during indexing operations
- **Directory exclusion**: `--exclude "node_modules"` and similar patterns now work correctly to exclude directories and files
- **Pattern matching**: Added support for gitignore-style glob patterns using ripgrep's `OverrideBuilder` for consistent, performant exclusion
- **Multiple exclusions**: Fixed support for multiple `--exclude` flags (e.g., `--exclude "node_modules" --exclude "*.log"`)

### Technical
- **ripgrep alignment**: Leveraged the `ignore` crate's `OverrideBuilder` for exclude pattern matching, aligning with ripgrep's proven approach
- **Streaming integration**: Exclude patterns now work correctly with the new streaming indexing architecture
- **API consistency**: Updated all indexing functions (`index_directory`, `smart_update_index`, etc.) to support exclude patterns

## [0.3.5] - 2025-09-07

### Added
- **Git integration**: Added support for respecting `.gitignore` files during search and indexing operations
- **Ignore control flag**: Added `--no-ignore` flag to disable gitignore support when needed
- **Clean implementation**: Uses the `ignore` crate for proper gitignore parsing and directory traversal

### Fixed
- **UTF-8 boundary panic**: Fixed panic when truncating text containing emojis or multi-byte UTF-8 characters in preview display

## [0.3.1] - 2025-09-06

### Improved
- **Enhanced UX for semantic search**: Added intelligent defaults (topk=10, threshold=0.6) for semantic search to reduce cognitive load
- **Better CLI discoverability**: Added `--limit` as intuitive alias for `--topk` flag
- **Improved help documentation**: Clear signposting of relevant flags with aligned messaging across examples and descriptions  
- **Informational output**: Semantic search now shows current parameters (e.g., "ℹ Semantic search: top 10 results, threshold ≥0.6")
- **Consistent flag documentation**: Help text now clearly shows defaults and relationships between flags

## [0.3.0] - 2025-09-06

### Fixed
- **Hybrid search indexing consistency**: Fixed hybrid search to use the same efficient v3 semantic indexing as semantic search mode, eliminating redundant index rebuilds and improving performance consistency
- **Directory validation**: Fixed issue where searching non-existent directories would silently fall back to parent directory indexes instead of showing clear error messages
- **Output stream separation**: All progress indicators and status messages now correctly output to stderr instead of stdout, ensuring clean output for piping and scripts
- **NaN sort handling**: Fixed edge cases with NaN values in similarity scoring that could cause inconsistent results

### Added
- **File listing flags**: Added grep-compatible `-l/--files-with-matches` and `-L/--files-without-matches` flags for listing filenames only
- **Enhanced visual output**: Implemented sophisticated match highlighting with color-coded similarity heatmaps using RGB gradients
- **Better user experience**: Added "No matches found" message to stderr when no results are found, improving clarity for users
- **Improved error handling**: Enhanced directory traversal error handling and graceful degradation for individual file failures
- **Incremental indexing**: Smart hash-based index updates that only reprocess changed files, dramatically improving index update performance

### Improved  
- **Indexing strategy optimization**: Smart embedding computation that only processes embeddings when needed for semantic/hybrid search, dramatically improving performance for regex-only workflows
- **Semantic search v3**: New implementation using pre-computed embeddings from sidecar files with span-based content extraction
- **Test infrastructure**: Enhanced integration tests with better binary path resolution and more resilient semantic search testing
- **Code quality**: Removed unused code, fixed compiler warnings, and improved error messaging throughout the codebase

## [0.2.0] - 2025-08-30

### Added
- Major improvements to CLI functionality
- Full-section feature implementation (`--full-section` flag)
- Comprehensive testing suite (40+ tests)
- Smart exclusion patterns for Python virtual environments and build artifacts
- Installation script with PATH setup (`install.sh`)

### Fixed
- CLI flag conflict: changed `-h` to `--no-filename` to avoid help conflict
- Proper handling of files with no filename
- File exclusion functionality during index creation
- Enhanced semantic search to return complete code sections

### Improved
- Updated documentation (README.md, PRD.txt) to reflect current implementation status
- Marked milestones M0-M5 as completed in project roadmap

## [0.1.0] - Initial Release

### Added
- Initial version of ck project with core functionality
- Drop-in grep compatibility with semantic search capabilities
- Basic regex, semantic, lexical, and hybrid search modes
- JSON output format for agent-friendly integration
- File indexing and sidecar management system

--- CLAUDE.md ---
# Claude Development Guide

This file contains project-specific instructions for Claude and other AI agents working on the ck codebase.
Whenever you actually use ck and it does something unexpected, jot it down in a file could UNEXPECTED.md - supply what you ran, what you expected to happen, what happened instead.


## Release Process

### Version Tagging Convention

**IMPORTANT**: Tags follow the format `X.Y.Z` (NO `v` prefix) to match current standard:

```bash
# Correct format (current standard since 0.3.8+)
git tag 0.4.1
git tag 0.3.9

# Old format (deprecated, do not use)
git tag v0.3.4
```

Always check existing tags first: `git tag --sort=-version:refname`

### Pre-Commit Quality Checks

**ALWAYS** run these commands in order before any commit:

1. **Linting**: `cargo clippy` - Fix all warnings
2. **Formatting**: `cargo fmt` - Format all code  
3. **Testing**: `cargo test` - Ensure all tests pass

### Version Bump Process

When bumping versions:

1. **Update workspace version**: `Cargo.toml` (workspace level)
2. **Update ALL crate versions**: Use find/replace across all `Cargo.toml` files
   ```bash
   find . -name "Cargo.toml" -exec sed -i '' 's/version = "OLD"/version = "NEW"/g' {} \;
   ```
3. **Update documentation versions**: Check `PRD.txt` and other docs
4. **Update CHANGELOG.md**: Add comprehensive release notes (see format below)

### CHANGELOG.md Format

Always update CHANGELOG.md with new releases. Follow this structure:

```markdown
## [X.Y.Z] - YYYY-MM-DD

### Added
- **Feature name**: Clear user-facing description
- **Technical capability**: What it enables

### Fixed  
- **Bug description**: What was broken and how it's fixed
- **Performance issue**: Specific improvements made

### Technical
- **Implementation details**: For maintainers and contributors
- **Dependencies**: New dependencies added
```

### Development Notes

- **Test coverage**: Maintain comprehensive test coverage (currently 65+ tests)
- **Cross-platform**: Ensure features work on Windows, macOS, and Linux
- **Performance**: Consider impact on indexing and search performance
- **User experience**: Maintain grep compatibility and intuitive CLI design

### Common Patterns in this Codebase

- **Error handling**: Use `anyhow::Result` consistently
- **Async/await**: Tokio runtime for async operations  
- **Parallel processing**: Rayon for CPU-intensive tasks
- **File I/O**: Memory-mapped files for large data access
- **Configuration**: Workspace-level dependency management

### Quality Standards

- All clippy warnings must be resolved
- Code must be formatted with `cargo fmt`
- All tests must pass
- New features require comprehensive test coverage
- Breaking changes require major version bump
- --help reflects any new features
- README incorporates any new user features (e.g. flags etc)

--- PRD.txt ---
# PRD + Engineering Plan: **ck — Semantic Grep by Embedding**

---

## Overview

**ck (seek)** is a Rust-based drop-in replacement for `grep`, extended with semantic and vector search capabilities. It fingerprints files, stores span embeddings, and allows both regex and semantic queries over text/code.

Just as **uv** modernized `pip`, **ck** modernizes `grep`: fast, lightweight, agent-friendly, and extensible.

---

## Goals

* **Agent Friendly**: Output is consistent, simple, and structured (grep-like text OR JSON). Claude, GPT, and other LLMs can consume results without fragile parsing.
* **Drop-in Replacement**: Matches grep CLI flags and defaults; regex queries behave identically.
* **Hybrid Search**: Fuse regex/BM25 with embedding similarity via Reciprocal Rank Fusion (RRF).
* **Code-Aware Chunking**: Default span segmentation + language-aware chunkers (Python → TypeScript → others).
* **Rust-First Implementation**: High performance, portable, no runtime deps.
* **Modular & Extensible**: Pluggable ANN index backends, embedders, and chunkers.
* **Self-contained**: Sidecar `.ck/` directory at repo root, fully removable, reproducible, idempotent.

---

## Non-Goals

* Not a full IDE or static analyzer.
* Not distributed search (local-first focus).
* Not a binary/large file searcher (text/code focus only).

---

## Target Users

* **LLM Agents** — primary consumers, via JSON. Use `ck` to “see” code semantically.
* **Developers** — command-line users who need grep-like familiarity with semantic upgrades.

---

## Key Features

### 1. Drop-in grep compatibility

* Core flags supported: `-n`, `-C`, `-A`, `-B`, `-R`, `-i`, `-F`, `-w`, `-h`, `-H`.
* Context handling matches grep semantics: `-C N` sets symmetric context; `-A N`/`-B N` control after/before independently.
* Filename printing follows grep conventions and can be forced with `-H` or suppressed with `-h`.
* Regex searches provide deterministic ordering (by file path, then line number) and apply `--topk` after sorting.
* Exit codes align with grep: 0 if matches found, 1 if none, 2 on error.
* File listing flags: `-l`/`--files-with-matches` and `-L`/`--files-without-matches` implemented.
* Additional flags like `-v`, `-c`, `-q` are candidates for future milestones.

### 2. Span Fingerprinting & Indexing

* Sidecar `.ck/` at repo root, mirrors repo layout (`foo.rs` → `.ck/foo.rs.ck`).
* **File-level hashing** (blake3 or sha256).
* Lifecycle:

  * `ck --index path/` → build/update incrementally (only rebuild files whose hash changed).
  * `ck --reindex` → force index update before searching.
  * `ck --clean` → wipe sidecar fully (safe to rebuild).
  * Smart incremental updates: hash-based change detection for fast re-indexing.
* **CI Guidance**: Recommend **regen per build**; `.ck/` is a cache artifact, not checked into VCS.
* **Concurrency**: atomic replace via `.tmp` + rename, file-level granularity.

### 3. Code-Aware Chunking

* Default: overlapping spans (256–512 tokens, configurable).
* Language-aware chunkers via tree-sitter:

  * Python: functions, classes.
  * TypeScript: functions, exports, classes.
* Fallback: generic text spans (ensures all languages supported).

### 4. Hybrid Search

* Regex or semantic query (or both).
* Ranking via **Reciprocal Rank Fusion (RRF)** \[Cormack & Clarke 2009].
* Deterministic and robust across scoring sources.
* LLM/agent consumption:

  * Intelligent defaults for semantic search: top 10 results, threshold ≥0.6.
  * Optional `--topk N` or `--limit N` to bound results for agent payload size.
  * Configurable threshold filtering with mode-appropriate defaults.
* CLI examples:

  ```bash
  ck --sem "retry with exponential backoff" src/     # Uses defaults: top 10, threshold ≥0.6
  ck --regex "httpClient" src/
  ck --hybrid --limit 20 "connection pool"          # --limit is alias for --topk
  ```

### 5. Agent-Friendly Output

* Default (grep-like):

  ```
  src/retry.py:42: async def retry_with_backoff(...  # score=3.21
  ```
* JSON mode (`--json-v1`):

  ```json
  {
    "file": "src/retry.py",
    "span": {"byte_start": 12288, "byte_end": 12960, "line_start": 40, "line_end": 55},
    "lang": "python",
    "symbol": "retry_with_backoff",
    "score": 3.21,
    "signals": {"lex_rank": 10, "vec_rank": 5, "rrf_score": 0.82},
    "preview": "async def retry_with_backoff(...",
    "model": "BAAI/bge-small-en-v1.5"
  }
  ```
* NDJSON streaming mode for large repos.
* Schema versioned (`--json-v1`, future `--json-v2`).
* JSON schema checked into repo and validated in CI.

### 6. Rust-First Stack

* **Lexical search:** Tantivy (BM25).
* **ANN:**

  * Default: pure-Rust HNSW (lightweight, portable).
  * Optional: FAISS backend (feature-gated) for IVF/PQ/GPU.
* **Embeddings:**

  * Default: fastembed (CPU-fast ONNX models; e.g. BGE-small, MiniLM).
  * Optional: Candle backend (Rust-native inference).
  * Optional: API backend (OpenAI/HF Inference compatible).
  * Version-pinned models for reproducibility.
  * **Scope:** English-only baseline (multilingual deferred).
* **Chunking:** tree-sitter grammars + fallback generic spans.
* **Index storage:** memmap2 + serde/bincode sidecars.

### 7. Modular Embedder & ANN APIs

```rust
pub trait Embedder {
    fn id(&self) -> &'static str;
    fn dim(&self) -> usize;
    fn embed(&mut self, texts: &[String]) -> Result<Vec<Vec<f32>>>;
}

pub trait AnnIndex {
    fn build(vectors: &[Vec<f32>]) -> Result<Self> where Self: Sized;
    fn search(&self, q: &[f32], topk: usize) -> Vec<(u32, f32)>;
    fn add(&mut self, id: u32, v: &[f32]);
    fn save(&self, path: &Path) -> Result<()>;
    fn load(path: &Path) -> Result<Self> where Self: Sized;
}
```

* Incremental add/update supported (`ck add file.rs`).
* ANN backend chosen at build (hnsw default, faiss optional).
* Model registry (`models.toml`) + per-project config (`ck.toml`).

---

## User Stories

* As an agent, I can `ck --json "find exponential backoff logic"` and get structured spans I can edit or extend.
* As a dev, I can run `ck -n "httpClient" src/` and see identical results to grep.
* As a dev, I can run `ck --sem "database connection pool"` and discover functions not matching literal keywords.
* As a system, I can update `.ck` indexes incrementally after a file changes.

---

## Technical Requirements

* Language: Rust (2021+).
* Perf targets:

  * ✅ Index 1M LOC repo in < 2 minutes (parallelized).
  * ✅ Query top-100 in < 500ms.
  * ✅ Index size: ≤ 2× source (with quantization).
* Extensibility: 
  * ✅ New chunker < 500 LOC (tree-sitter integration demonstrated).
  * ✅ New embedder < 200 LOC (trait-based architecture).
* Testing:
  * ✅ Comprehensive test suite (40+ tests).
  * ✅ CI/CD validation of core functionality.

---

## Success Metrics — STATUS UPDATE

* ✅ **Drop-in adoption:** can replace `grep` in scripts with no breakage.
* ✅ **Agent usability:** JSON mode works out-of-box for Claude/GPT.
* ✅ **Performance:** sub-500ms queries, lightweight indexes.
* ✅ **Extensibility:** new chunker < 500 LOC; new embedder < 200 LOC.
* ✅ **Index hygiene:** `ck clean && ck index` → reproducible results.
* ✅ **Production readiness:** Comprehensive testing, installation scripts, proper exclusions.
* ✅ **Code-aware features:** `--full-section` flag for complete semantic units.

---

## Cargo Workspace Layout

```
ck/
├── ck-cli/       # end-user binary
├── ck-core/      # shared core logic
├── ck-index/     # index builder (sidecars, hashes)
├── ck-search/    # query engine (BM25 + ANN + RRF fusion)
├── ck-chunk/     # span segmentation + tree-sitter
├── ck-embed/     # embedders (fastembed, Candle, API)
├── ck-ann/       # ANN backends (HNSW, FAISS)
└── ck-models/    # model registry + config
```

---

## Milestones

### **M0 — Foundations** ✅ COMPLETED

* ✅ Workspace scaffolding.
* ✅ Regex-focused CLI with grep parity for core flags (`-n`, `-C/-A/-B`, `-R`, `-i`, `-F`, `-w`, `--no-filename`, `-H`).
* ✅ Deterministic output ordering and grep-like exit codes.
* ✅ `.ck/` sidecar spec defined (file-level hash).
* ✅ Comprehensive test suite (40+ tests) validates grep parity.

### **M1 — Lexical Search** ✅ COMPLETED

* ✅ Tantivy BM25 integration.
* ✅ CLI: `ck --lex query`.
* ✅ Respects `recursive` flag consistently and reuses nearest existing `.ck` index by walking up from the search path.

### **M2 — Semantic Embeddings** ✅ COMPLETED

* ✅ Fastembed integration (feature-gated, dummy fallback for CI).
* ✅ Basic ANN backend (SimpleIndex with cosine similarity).
* ✅ CLI: `ck --sem query`.
* ✅ Respects `recursive` flag consistently and reuses nearest existing `.ck` index by walking up from the search path.
* ⚠️ Note: Uses basic brute-force search, not HNSW (optimization needed for large codebases).

### **M3 — Hybrid Search** ✅ COMPLETED

* ✅ Reciprocal Rank Fusion integration.
* ✅ `--topk` flag for controlling payload size (applied after deterministic sort for regex results).
* ✅ Deterministic mode for CI.
* ✅ `--threshold` flag for score filtering.

### **M4 — Code-Aware Chunking** ✅ COMPLETED

* ✅ Overlapping spans + tree-sitter Python/TypeScript/JavaScript.
* ✅ Fallback chunker for unsupported languages.
* ✅ `--full-section` flag to return complete functions/classes.
* ✅ Semantic boundary detection for function, class, method extraction.

### **M5 — Agent-Friendly Output** ✅ COMPLETED

* ✅ JSON v1 schema (stable, versioned).
* ✅ Output format remains grep-like for text mode; JSON includes scores and spans.
* ✅ `--json` flag for structured output.
* ✅ `--scores` flag to show relevance scores.
* 🚧 NDJSON streaming (not yet implemented).

### **M6 — Extensibility & Cleanup** ✅ COMPLETED

* ✅ `ck --clean` + robust lifecycle.
* ✅ Index management (`--status`, `--add`, `--clean-orphans`).
* ✅ Smart exclusion patterns (build artifacts, virtual environments).
* ✅ File listing flags (`-l`/`-L` for grep compatibility).
* ✅ Incremental indexing with hash-based change detection.
* ✅ Enhanced UX with intelligent defaults and informational output.
* 🚧 FAISS + Candle optional backends (deferred).
* 🚧 Model registry + per-project `ck.toml` (deferred).

---

## Implementation Status & Decisions

* ✅ Simple ANN index implemented (HNSW/FAISS deferred for performance optimization).
* ✅ fastembed integration (feature-gated for CI compatibility).
* ✅ JSON schema versioning (`--json` flag).
* ✅ Sidecar `.ck/` is cache-only, not committed to VCS.
* ✅ Hybrid fusion via Reciprocal Rank Fusion (RRF).
* ✅ File-level hash indexing; reindex on-demand with `--reindex`.
* ✅ Exclusion patterns with comprehensive defaults (`.git`, `node_modules`, `.venv`, `__pycache__`, etc.).
* ✅ `--no-default-excludes` disables default exclusions.
* ✅ CLI flag conflict resolved (`-h` → `--no-filename`).
* ✅ Tree-sitter semantic chunking with `--full-section` support.
* ✅ File listing flags (`-l`/`-L`) for grep-compatible file enumeration.
* ✅ Incremental indexing with smart hash-based change detection.
* ✅ Enhanced UX: intelligent defaults, `--limit` alias, informational output.
* ✅ Clean stdout/stderr separation for reliable scripting and piping.

---

## Current Implementation Status (v0.4.4)

### ✅ **Production Ready Features**
- **Complete grep replacement**: All major flags supported with identical behavior
- **File listing compatibility**: `-l`/`-L` flags for grep-compatible file enumeration
- **Semantic search**: Works with FastEmbed integration and intelligent defaults (top 10, threshold ≥0.6)
- **Incremental indexing**: Hash-based change detection for fast index updates  
- **Index management**: Create, update, clean, status, add single files, orphan cleanup
- **Smart exclusions**: Automatically skips virtual environments, build artifacts
- **Multiple search modes**: regex (default), semantic, lexical (BM25), hybrid (RRF)
- **Enhanced UX**: Intuitive `--limit` alias, informational output, clear help documentation
- **Agent-friendly output**: JSON format with scores and spans, clean stdout/stderr separation
- **Tree-sitter integration**: Python, JavaScript, TypeScript parsing
- **Code section extraction**: `--full-section` flag for complete functions/classes
- **Comprehensive testing**: 40+ tests covering all functionality
- **Easy installation**: Automated install script with PATH setup

### ⚠️ **Known Limitations** 
- **ANN Performance**: Uses brute-force cosine similarity (sufficient for most use cases, HNSW/FAISS deferred)
- **Configuration**: No per-project config files yet (CLI flags provide full control)
- **Language Support**: Tree-sitter limited to Python/JS/TS (easily expandable)

### 🎯 **Future Considerations** (Post v1.0)
1. **Performance Optimization**: HNSW or FAISS backends for very large codebases
2. **Configuration System**: Support `.ck/config.toml` for per-project settings  
3. **Language Expansion**: Add Rust, Go, Java tree-sitter support based on demand
4. **Package Distribution**: Homebrew, cargo install, apt packages for easier installation
5. **Advanced Features**: Multi-language semantic understanding, IDE integrations

---

## Branding

**ck — seek code, semantically.**
c = see, k = seek.
Unix-y minimalism; short, verb-friendly: "just ck it."

---


--- README.md ---
# ck - Semantic Code Search

[![CI](https://github.com/BeaconBay/ck/actions/workflows/ci.yaml/badge.svg)](https://github.com/BeaconBay/ck/actions/workflows/ci.yaml)
[![Crates.io](https://img.shields.io/crates/v/ck-search.svg)](https://crates.io/crates/ck-search)
[![Downloads](https://img.shields.io/crates/d/ck-search.svg)](https://crates.io/crates/ck-search)
[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)](LICENSE-MIT)
[![MSRV](https://img.shields.io/badge/rust-1.88%2B-blue.svg)](https://www.rust-lang.org)
[![Documentation](https://img.shields.io/badge/docs-beaconbay.github.io%2Fck-blue)](https://beaconbay.github.io/ck/)

**ck (seek)** finds code by meaning, not just keywords. It's grep that understands what you're looking for — search for "error handling" and find try/catch blocks, error returns, and exception handling code even when those exact words aren't present.

## 🚀 Quick Start

```bash
# Install from crates.io
cargo install ck-search

# Just search — ck builds and updates indexes automatically
ck --sem "error handling" src/
ck --sem "authentication logic" src/
ck --sem "database connection pooling" src/

# Traditional grep-compatible search still works
ck -n "TODO" *.rs
ck -R "TODO|FIXME" .

# Combine both: semantic relevance + keyword filtering
ck --hybrid "connection timeout" src/
```

> **📚 [Full Documentation](https://beaconbay.github.io/ck/)** — Installation guides, tutorials, feature deep-dives, and API reference

## ✨ Headline Features

### 🤖 **AI Agent Integration (MCP Server)**
Connect ck directly to Claude Desktop, Cursor, or any MCP-compatible AI client for seamless code search integration:

```bash
# Start MCP server for AI agent integration
ck --serve
```

**Claude Desktop Setup:**

```bash
# Install via Claude Code CLI (recommended)
claude mcp add ck-search -s user -- ck --serve

# Note: You may need to restart Claude Code after installation
# Verify installation with:
claude mcp list  # or use /mcp in Claude Code
```

**Manual Configuration (alternative):**
```json
{
  "mcpServers": {
    "ck": {
      "command": "ck",
      "args": ["--serve"],
      "cwd": "/path/to/your/codebase"
    }
  }
}
```

**Tool Permissions:** When prompted by Claude Code, approve permissions for ck-search tools (semantic_search, regex_search, hybrid_search, etc.)

**Available MCP Tools:**
- `semantic_search` - Find code by meaning using embeddings
- `regex_search` - Traditional grep-style pattern matching
- `hybrid_search` - Combined semantic and keyword search
- `index_status` - Check indexing status and metadata
- `reindex` - Force rebuild of search index
- `health_check` - Server status and diagnostics

**Built-in Pagination:** Handles large result sets gracefully with page_size controls, cursors, and snippet length management.

### 🎨 **Interactive TUI (Terminal User Interface)**
Launch an interactive search interface with real-time results and multiple preview modes:

```bash
# Start TUI for current directory
ck --tui

# Start with initial query
ck --tui "error handling"
```

**Features:**
- **Multiple Search Modes**: Toggle between Semantic, Regex, and Hybrid search with `Tab`
- **Preview Modes**: Switch between Heatmap, Syntax highlighting, and Chunk view with `Ctrl+V`
- **View Options**: Toggle between snippet and full-file view with `Ctrl+F`
- **Multi-select**: Select multiple files with `Ctrl+Space`, open all in editor with `Enter`
- **Search History**: Navigate with `Ctrl+Up/Down`
- **Editor Integration**: Opens files in `$EDITOR` with line numbers (Vim, VS Code, Cursor, etc.)
- **Progress Tracking**: Live indexing progress with file and chunk counts
- **Config Persistence**: Preferences saved to `~/.config/ck/tui.json`

See [TUI.md](TUI.md) for keyboard shortcuts and detailed usage.

### 🔍 **Semantic Search**
Find code by concept, not keywords. Understands synonyms, related terms, and conceptual similarity:

```bash
# These find related code even without exact keywords:
ck --sem "retry logic"           # finds backoff, circuit breakers
ck --sem "user authentication"   # finds login, auth, credentials
ck --sem "data validation"       # finds sanitization, type checking

# Get complete functions/classes containing matches
ck --sem --full-section "error handling"  # returns entire functions
```

### ⚡ **Drop-in grep Compatibility**
All your muscle memory works. Same flags, same behavior, same output format:

```bash
ck -i "warning" *.log              # Case-insensitive
ck -n -A 3 -B 1 "error" src/       # Line numbers + context
ck -l "error" src/                  # List files with matches only
ck -L "TODO" src/                   # List files without matches
ck -R --exclude "*.test.js" "bug"  # Recursive with exclusions
```

### 🎯 **Hybrid Search**
Combine keyword precision with semantic understanding using Reciprocal Rank Fusion:

```bash
ck --hybrid "async timeout" src/    # Best of both worlds
ck --hybrid --scores "cache" src/   # Show relevance scores with color highlighting
ck --hybrid --threshold 0.02 query  # Filter by minimum relevance
```

### ⚙️ **Automatic Delta Indexing with Chunk-Level Caching**
Semantic and hybrid searches transparently create and refresh their indexes before running. The first search builds what it needs; subsequent searches intelligently reuse cached embeddings:

- **Chunk-level incremental indexing**: Only changed chunks are re-embedded (80-90% cache hit rate for typical code changes)
- **Content-aware invalidation**: Doc comments and whitespace changes properly invalidate cache
- **Model consistency**: Prevents silent embedding corruption when switching models
- **Smart caching**: Hash-based invalidation using blake3(text + trivia) for reliable change detection

### 📁 **Smart File Filtering**
Automatically excludes cache directories, build artifacts, and respects `.gitignore` and `.ckignore` files:

```bash
# ck respects multiple exclusion layers (all are additive):
ck "pattern" .                           # Uses .gitignore + .ckignore + defaults
ck --no-ignore "pattern" .               # Skip .gitignore (still uses .ckignore)
ck --no-ckignore "pattern" .             # Skip .ckignore (still uses .gitignore)
ck --exclude "dist" --exclude "logs" .   # Add custom exclusions

# .ckignore file (created automatically on first index):
# - Excludes images, videos, audio, binaries, archives by default
# - Excludes JSON/YAML config files (issue #27)
# - Uses same syntax as .gitignore (glob patterns, ! for negation)
# - Persists across searches (issue #67)
# - Located at repository root, editable for custom patterns

# Exclusion patterns use .gitignore syntax:
ck --exclude "node_modules" .            # Exclude directory and all contents
ck --exclude "*.test.js" .                # Exclude files matching pattern
ck --exclude "build/" --exclude "*.log" . # Multiple exclusions
# Note: Patterns are relative to the search root
```

**Why .ckignore?** While `.gitignore` handles version control exclusions, many files that *should* be in your repo aren't ideal for semantic search. Config files (`package.json`, `tsconfig.json`), images, videos, and data files add noise to search results and slow down indexing. `.ckignore` lets you focus semantic search on actual code while keeping everything else in git. Think of it as "what should I search" vs "what should I commit".

## 🛠 Advanced Usage

### AI Agent Integration

#### MCP Server (Recommended)
```python
# Example usage in AI agents
response = await client.call_tool("semantic_search", {
    "query": "authentication logic",
    "path": "/path/to/code",
    "page_size": 25,
    "top_k": 50,           # Limit total results (default: 100 for MCP)
    "snippet_length": 200
})

# Handle pagination
if response["pagination"]["next_cursor"]:
    next_response = await client.call_tool("semantic_search", {
        "query": "authentication logic",
        "path": "/path/to/code",
        "cursor": response["pagination"]["next_cursor"]
    })
```

#### JSONL Output (Custom Workflows)
Perfect structured output for LLMs, scripts, and automation:

```bash
# JSONL format - one JSON object per line (recommended for agents)
ck --jsonl --sem "error handling" src/
ck --jsonl --no-snippet "function" .        # Metadata only
ck --jsonl --topk 5 --threshold 0.7 "auth"  # High-confidence results

# Traditional JSON (single array)
ck --json --sem "error handling" src/ | jq '.file'
```

**Why JSONL for AI agents?**
- ✅ **Streaming friendly**: Process results as they arrive
- ✅ **Memory efficient**: Parse one result at a time
- ✅ **Error resilient**: One malformed line doesn't break entire response
- ✅ **Standard format**: Used by OpenAI API, Anthropic API, and modern ML pipelines

### Search & Filter Options

```bash
# Threshold filtering
ck --sem --threshold 0.7 "query"           # Only high-confidence matches
ck --hybrid --threshold 0.01 "concept"     # Low-confidence (exploration)

# Limit results
ck --sem --topk 5 "authentication patterns"

# Complete code sections
ck --sem --full-section "database queries"  # Complete functions
ck --full-section "class.*Error" src/       # Complete classes (works with regex too)

# Relevance scoring
ck --sem --scores "machine learning" docs/
# [0.847] ./ai_guide.txt: Machine learning introduction...
# [0.732] ./statistics.txt: Statistical learning methods...
```


### Language Coverage

| Language | Indexing | Chunking | AST-aware | Notes |
|----------|----------|----------|-----------|-------|
| Zig | ✅ | ✅ | ✅ | contributed by [@Nevon](https://github.com/Nevon) (PR #72) |

### Model Selection

Choose the right embedding model for your needs:

```bash
# Default: BGE-Small (fast, precise chunking)
ck --index .

# Enhanced: Nomic V1.5 (8K context, optimal for large functions)
ck --index --model nomic-v1.5 .

# Code-specialized: Jina Code (optimized for programming languages)
ck --index --model jina-code .
```

**Model Comparison:**
- **`bge-small`** (default): 400-token chunks, fast indexing, good for most code
- **`nomic-v1.5`**: 1024-token chunks with 8K model capacity, better for large functions
- **`jina-code`**: 1024-token chunks with 8K model capacity, specialized for code understanding

### Index Management

```bash
# Check index status
ck --status .

# Clean up and rebuild / switch models
ck --clean .
ck --switch-model nomic-v1.5 .
ck --switch-model nomic-v1.5 --force .     # Force rebuild

# Add single file to index
ck --add new_file.rs

# File inspection (analyze chunking and token usage)
ck --inspect src/main.rs
ck --inspect --model bge-small src/main.rs  # Test different models
```

**Interrupting Operations:** Indexing can be safely interrupted with Ctrl+C. The partial index is saved, and the next operation will resume from where it stopped, only processing new or changed files.

## 📚 Language Support

| Language | Indexing | Tree-sitter Parsing | Semantic Chunking |
|----------|----------|-------------------|------------------|
| Python | ✅ | ✅ | ✅ Functions, classes |
| JavaScript/TypeScript | ✅ | ✅ | ✅ Functions, classes, methods |
| Rust | ✅ | ✅ | ✅ Functions, structs, traits |
| Go | ✅ | ✅ | ✅ Functions, types, methods |
| Ruby | ✅ | ✅ | ✅ Classes, methods, modules |
| Haskell | ✅ | ✅ | ✅ Functions, types, instances |
| C# | ✅ | ✅ | ✅ Classes, interfaces, methods |

**Text Formats:** Markdown, JSON, YAML, TOML, XML, HTML, CSS, shell scripts, SQL, log files, config files, and any other text format.

**Smart Binary Detection:** Uses ripgrep-style content analysis, automatically indexing any text file while correctly excluding binary files.

**Unsupported File Types:** Text files with unrecognized extensions (like `.org`, `.adoc`, etc.) are automatically indexed as plain text. ck detects text vs binary based on file contents, not extensions.

## 🏗 Installation

### From crates.io
```bash
cargo install ck-search
```

### From Source
```bash
git clone https://github.com/BeaconBay/ck
cd ck
cargo install --path ck-cli
```

### Package Managers
```bash
# Currently available:
cargo install ck-search    # ✅ Available now via crates.io

# Coming soon:
brew install ck-search     # 🚧 In development (use cargo for now)
apt install ck-search      # 🚧 In development
```

## 💡 Examples

### Finding Code Patterns
```bash
# Find authentication/authorization code
ck --sem "user permissions" src/
ck --sem "access control" src/
ck --sem "login validation" src/

# Find error handling strategies
ck --sem "exception handling" src/
ck --sem "error recovery" src/
ck --sem "fallback mechanisms" src/

# Find performance-related code
ck --sem "caching strategies" src/
ck --sem "database optimization" src/
ck --sem "memory management" src/
```

### Team Workflows
```bash
# Find related test files
ck --sem "unit tests for authentication" tests/
ck -l --sem "test" tests/           # List test files by semantic content

# Identify refactoring candidates
ck --sem "duplicate logic" src/
ck --sem "code complexity" src/
ck -L "test" src/                   # Find source files without tests

# Security audit
ck --hybrid "password|credential|secret" src/
ck --sem "input validation" src/
```

### Integration Examples
```bash
# Git hooks
git diff --name-only | xargs ck --sem "TODO"

# CI/CD pipeline
ck --json --sem "security vulnerability" . | security_scanner.py

# Code review prep
ck --hybrid --scores "performance" src/ > review_notes.txt

# Documentation generation
ck --json --sem "public API" src/ | generate_docs.py
```

## ⚡ Performance

**Field-tested on real codebases:**

- **Indexing:** ~1M LOC in under 2 minutes
- **Incremental indexing:** 80-90% cache hit rate for typical code changes (only changed chunks re-embedded)
- **Search:** Sub-500ms queries on typical codebases
- **Index size:** ~2x source code size with compression
- **Memory:** Efficient streaming for large repositories
- **Token precision:** HuggingFace tokenizers for exact model-specific token counting

## 🔧 Architecture

ck uses a modular Rust workspace:

- **`ck-cli`** - Command-line interface and MCP server
- **`ck-tui`** - Interactive terminal user interface (ratatui-based)
- **`ck-core`** - Shared types, configuration, and utilities
- **`ck-engine`** - Search engine implementations (regex, semantic, hybrid)
- **`ck-index`** - File indexing, hashing, and sidecar management
- **`ck-embed`** - Text embedding providers (FastEmbed, API backends)
- **`ck-ann`** - Approximate nearest neighbor search indices
- **`ck-chunk`** - Text segmentation and language-aware parsing ([query-based chunking](docs/QUERY_BASED_CHUNKING.md))
- **`ck-models`** - Model registry and configuration management

### Index Storage

Indexes are stored in `.ck/` directories alongside your code:

```
project/
├── src/
├── docs/
└── .ck/           # Semantic index (can be safely deleted)
    ├── embeddings.json
    ├── ann_index.bin
    └── tantivy_index/
```

The `.ck/` directory is a cache — safe to delete and rebuild anytime.

## 🧪 Testing

```bash
# Run the full test suite
cargo test --workspace

# Test with each feature combination
cargo hack test --each-feature --workspace
```

## 🤝 Contributing

ck is actively developed and welcomes contributions:

1. **Issues:** Report bugs, request features
2. **Code:** Submit PRs for bug fixes, new features
3. **Documentation:** Improve examples, guides, tutorials
4. **Testing:** Help test on different codebases and languages

### Development Setup
```bash
git clone https://github.com/BeaconBay/ck
cd ck
cargo build --workspace
cargo test --workspace
./target/debug/ck --index test_files/
./target/debug/ck --sem "test query" test_files/
```

### CI Requirements
Before submitting a PR, ensure your code passes all CI checks:

```bash
# Format code (required)
cargo fmt --all

# Run clippy linter (required - must have no warnings)
cargo clippy --workspace --all-features --all-targets -- -D warnings

# Run tests (required)
cargo test --workspace

# Check minimum supported Rust version (MSRV)
cargo hack check --each-feature --locked --rust-version --workspace
```

The CI pipeline runs on Ubuntu, Windows, and macOS to ensure cross-platform compatibility.

## 🗺 Roadmap

### Current (v0.7+)
- ✅ MCP (Model Context Protocol) server for AI agent integration
- ✅ Chunk-level incremental indexing with smart embedding reuse
- ✅ grep-compatible CLI with semantic search and file listing flags
- ✅ FastEmbed integration with BGE models and enhanced model selection
- ✅ File exclusion patterns and glob support
- ✅ Threshold filtering and relevance scoring with visual highlighting
- ✅ Tree-sitter parsing and intelligent chunking for 7+ languages
- ✅ Complete code section extraction (`--full-section`)
- ✅ Clean stdout/stderr separation for reliable scripting
- ✅ Token-aware chunking with HuggingFace tokenizers
- ✅ Published to crates.io (`cargo install ck-search`)

### Next (v0.6+)
- 🚧 Configuration file support
- 🚧 Package manager distributions (brew, apt)
- 🚧 Enhanced MCP tools (file writing, refactoring assistance)
- 🚧 VS Code extension
- 🚧 JetBrains plugin
- 🚧 Additional language chunkers (Java, PHP, Swift)

## ❓ FAQ

**Q: How is this different from grep/ripgrep/silver-searcher?**
A: ck includes all the features of traditional search tools, but adds semantic understanding. Search for "error handling" and find relevant code even when those exact words aren't used.

**Q: Does it work offline?**
A: Yes, completely offline. The embedding model runs locally with no network calls.

**Q: How big are the indexes?**
A: Typically 1-3x the size of your source code. The `.ck/` directory can be safely deleted to reclaim space.

**Q: Is it fast enough for large codebases?**
A: Yes. The first semantic search builds the index automatically; after that only changed files are reprocessed, keeping searches sub-second even on large projects.

**Q: Can I use it in scripts/automation?**
A: Absolutely. The `--json` and `--jsonl` flags provide structured output perfect for automated processing and AI agent integration.

**Q: What about privacy/security?**
A: Everything runs locally. No code or queries are sent to external services. The embedding model is downloaded once and cached locally.

**Q: Where are the embedding models cached?**
A: Models are cached in platform-specific directories:
- Linux/macOS: `~/.cache/ck/models/`
- Windows: `%LOCALAPPDATA%\ck\cache\models\`
- Fallback: `.ck_models/models/` in current directory

## 📄 License

Licensed under either of:
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))
- MIT License ([LICENSE-MIT](LICENSE-MIT))

at your option.

## 🙏 Credits

Built with:
- [Rust](https://rust-lang.org) - Systems programming language
- [FastEmbed](https://github.com/Anush008/fastembed-rs) - Fast text embeddings
- [Tantivy](https://github.com/quickwit-oss/tantivy) - Full-text search engine
- [clap](https://github.com/clap-rs/clap) - Command line argument parsing

Inspired by the need for better code search tools in the age of AI-assisted development.

---

**Start finding code by what it does, not what it says.**

```bash
cargo install ck-search
ck --sem "the code you're looking for"
```


## Links discovered
- [![CI](https://github.com/BeaconBay/ck/actions/workflows/ci.yaml/badge.svg)
- [![Crates.io](https://img.shields.io/crates/v/ck-search.svg)
- [![Downloads](https://img.shields.io/crates/d/ck-search.svg)
- [![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue.svg)
- [![MSRV](https://img.shields.io/badge/rust-1.88%2B-blue.svg)
- [![Documentation](https://img.shields.io/badge/docs-beaconbay.github.io%2Fck-blue)
- [Full Documentation](https://beaconbay.github.io/ck/)
- [TUI.md](https://github.com/BeaconBay/ck/blob/main/TUI.md)
- [@Nevon](https://github.com/Nevon)
- [query-based chunking](https://github.com/BeaconBay/ck/blob/main/docs/QUERY_BASED_CHUNKING.md)
- [LICENSE-APACHE](https://github.com/BeaconBay/ck/blob/main/LICENSE-APACHE.md)
- [LICENSE-MIT](https://github.com/BeaconBay/ck/blob/main/LICENSE-MIT.md)
- [Rust](https://rust-lang.org)
- [FastEmbed](https://github.com/Anush008/fastembed-rs)
- [Tantivy](https://github.com/quickwit-oss/tantivy)
- [clap](https://github.com/clap-rs/clap)

--- TUI.md ---
# ck TUI (Terminal User Interface)

The ck TUI provides an interactive search interface with real-time results, multiple preview modes, and keyboard-driven navigation.

## Quick Start

```bash
# Launch TUI in current directory
ck --tui

# Launch with initial query
ck --tui "error handling"

# Launch in specific directory
ck --tui --path /path/to/code
```

## Keyboard Shortcuts

### Navigation
| Key | Action |
|-----|--------|
| `↑` / `↓` | Navigate through search results |
| `PageUp` / `PageDown` | Scroll preview (in full-file mode) |
| `Enter` | Open selected file(s) in `$EDITOR` |
| `Ctrl+Up` / `Ctrl+Down` | Navigate search history |

### Search & Modes
| Key | Action |
|-----|--------|
| `Tab` | Cycle search modes (Semantic → Regex → Hybrid) |
| Type any text | Update search query (300ms debounce) |
| `Backspace` | Delete character from query |
| `/command` | Enter command mode (see Commands below) |

### View Controls
| Key | Action |
|-----|--------|
| `Ctrl+V` | Cycle preview modes (Heatmap → Syntax → Chunks) |
| `Ctrl+F` | Toggle snippet/full-file view |
| `Ctrl+D` | Show chunk metadata for current file |

### Multi-Select
| Key | Action |
|-----|--------|
| `Ctrl+Space` | Toggle selection of current file |
| `Enter` | Open all selected files (or current if none selected) |

### Exit
| Key | Action |
|-----|--------|
| `q` or `Esc` | Quit TUI |
| `Ctrl+C` | Force quit |

## Search Modes

### Semantic Search (Default)
Finds code by meaning using embeddings. Best for conceptual queries:
```
"error handling"
"database connection pooling"
"user authentication"
```

### Regex Search
Traditional pattern matching for exact text searches:
```
"TODO|FIXME"
"function.*Error"
"\bauth\b"
```

### Hybrid Search
Combines semantic understanding with keyword precision using Reciprocal Rank Fusion:
```
"async timeout"
"cache invalidation"
```

## Preview Modes

### Heatmap Mode (Default)
Shows semantic similarity with color-coded highlighting:
- **Red**: Lower similarity (0.6-0.7)
- **Yellow**: Medium similarity (0.7-0.85)
- **Green**: High similarity (0.85+)

### Syntax Mode
Displays syntax-highlighted code using your file's language:
- Powered by `syntect` for accurate highlighting
- Supports 7+ languages plus text formats

### Chunks Mode
Shows chunk boundaries and metadata:
- Visual indicators for chunk start/end
- Chunk type annotations (Function, Class, Method)
- Useful for understanding how code is indexed

## View Options

### Snippet View (Default)
Shows ±5 lines of context around matches. Perfect for quick scanning of results.

### Full File View
Press `Ctrl+F` to toggle. Features:
- Scrollable full file content
- Auto-scrolls to matched line when navigating results
- `PageUp`/`PageDown` for navigation
- Matched line stays highlighted

## Command Mode

Start your query with `/` to enter command mode:

```
/open <filename>    - Open specific file
/config             - Show current configuration
/help               - Show help message
```

## Multi-Select Workflow

1. Navigate to a file you want to open (`↑` / `↓`)
2. Press `Ctrl+Space` to select it
3. Continue selecting additional files
4. Press `Enter` to open all selected files

Selected files are shown with a `[✓]` indicator in the results list.

## Editor Integration

The TUI opens files in your `$EDITOR` (or `$VISUAL`) with line numbers. Supported editors:

| Editor | Support | Format |
|--------|---------|--------|
| Vim/Neovim | ✅ Full | Opens in tabs with line numbers |
| VS Code | ✅ Full | `-g file:line` format |
| Cursor | ✅ Full | `-g file:line` format |
| Sublime | ✅ Full | `file:line` format |
| Emacs | ✅ Limited | Opens first file only |
| Nano | ✅ Limited | Opens first file only |

Set your editor:
```bash
export EDITOR=vim
export EDITOR=code
export EDITOR=cursor
```

## Progress Tracking

The TUI shows detailed progress during indexing:

```
Indexing repository for semantic search...
src/main.rs • 23/145 files • 5/12 chunks
[████████████░░░░░░░░] 67%
```

- **File name**: Currently processing file
- **File progress**: Completed/total files
- **Chunk progress**: Completed/total chunks in current file
- **Progress bar**: Overall completion percentage

## Configuration

TUI preferences are automatically saved to:
- **Linux/macOS**: `~/.config/ck/tui.json`
- **Windows**: `%APPDATA%\ck\tui.json`

Saved settings:
- Last used search mode
- Preview mode preference
- Full-file mode setting

## Search History

The TUI maintains a history of your last 20 searches:
- Navigate with `Ctrl+Up` / `Ctrl+Down`
- History persists across searches within same session
- Duplicate queries are not added

## Performance Tips

1. **First search**: May take longer as index is built
2. **Subsequent searches**: Near-instant using cached index
3. **Large codebases**: Use specific query terms to reduce result set
4. **Snippet mode**: Faster rendering than full-file mode

## Troubleshooting

### TUI won't start
- Ensure terminal supports colors: `echo $TERM`
- Check terminal size: `stty size` (minimum 80x24)

### Editor doesn't open
- Verify `$EDITOR` is set: `echo $EDITOR`
- Ensure editor is in `$PATH`

### No search results
- Check search mode (Tab to cycle)
- Try broader query terms
- Verify files are indexed: `ck --status .`

### Slow performance
- First search builds index (one-time cost)
- Large result sets: Add threshold with `--threshold 0.7`
- Check index size: `du -sh .ck/`

## Examples

### Find authentication code
```bash
ck --tui "authentication"
# Press Tab to cycle to Regex mode
# Type: "login|auth|credentials"
```

### Review error handling
```bash
ck --tui "error handling"
# Use Ctrl+Space to select multiple files
# Press Enter to open all in editor
```

### Explore code structure
```bash
ck --tui "database"
# Press Ctrl+V to switch to Chunks mode
# Press Ctrl+F for full-file view
# Use PageDown to explore
```

## Architecture

The TUI is implemented in the `ck-tui` crate:
- **app.rs** (988 lines): Main event loop and state management
- **preview.rs** (658 lines): Preview rendering (heatmap, syntax, chunks)
- **chunks.rs** (428 lines): Chunk display and metadata
- **commands.rs** (408 lines): Command mode execution
- **rendering.rs** (210 lines): UI component rendering
- **state.rs** (45 lines): Application state
- **config.rs** (90 lines): Configuration persistence

Built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).

## Contributing

The TUI is under active development. Contributions welcome for:
- Additional preview modes
- Custom keyboard shortcuts
- Theme/color customization
- Performance optimizations

See [CONTRIBUTING.md](CONTRIBUTING.md) for development setup.


## Links discovered
- [ratatui](https://github.com/ratatui-org/ratatui)
- [crossterm](https://github.com/crossterm-rs/crossterm)
- [CONTRIBUTING.md](https://github.com/BeaconBay/ck/blob/main/CONTRIBUTING.md)

--- UNEXPECTED.md ---
# Unexpected Behaviors in ck

This file tracks instances where ck behaves unexpectedly during testing or usage.

## Format

**Command:** `command run`  
**Expected:** What should have happened  
**Actual:** What actually happened  
**Date:** YYYY-MM-DD  
**Status:** [Fixed/Open/Investigating]

---

## Issues Found

**Command:** `ck ""`
**Expected:** Error message or no results
**Actual:** Massive output with every empty line in the codebase being matched
**Date:** 2025-09-13
**Status:** Open
**Notes:** Empty pattern matches all empty lines, resulting in overwhelming output. Perhaps should warn or limit results?

---

**Command:** `ck --add /tmp/test.txt`
**Expected:** Add file to index or meaningful error
**Actual:** Error: "No file specified. Usage: ck --add <file>"
**Date:** 2025-09-13
**Status:** Open
**Notes:** The error message is incorrect - a file was specified. Seems like argument parsing issue.

---

**Command:** `ck --sem "🎉🦀✨"`
**Expected:** No results or error about emoji patterns
**Actual:** Returns seemingly random code results
**Date:** 2025-09-13
**Status:** Open
**Notes:** Emoji search returns unrelated results. The semantic embedding seems to handle emojis unpredictably.

---

## Instructions

When you encounter unexpected behavior while using ck:

1. Note the exact command you ran
2. Describe what you expected to happen
3. Describe what actually happened
4. Add the date
5. Set status to "Open"

This helps track and fix edge cases and user experience issues.

--- vscode_prd.md ---
Product Requirements Document (PRD)
Product: ck for VS Code / Cursor
Version: v1.0
Author: Mike (Beacon Bay)
Date: October 2025
1. Overview
ck is a hybrid semantic + BM25 code search engine with blazing-fast delta indexing and optional MCP server mode.
The VS Code / Cursor extension brings ck’s command-line power and semantic awareness into the editor itself — combining:
TUI-style search interaction (fast keyboard UX, dense visual layout)
Rich semantic results (ranked by meaning, not just text)
Tight integration with code navigation and context (jump, peek, reindex)
It replaces the need to alt-tab to the terminal or separate UIs. Developers can query, browse, and open results entirely within VS Code or Cursor.
2. Goals & Non-Goals
Goals
Create a fast, tactile in-editor search experience that feels like a TUI but fits VS Code’s sidebar paradigm.
Provide two integration modes:
CLI Mode: spawn the local ck binary (ck --hybrid "query").
MCP Mode: connect to a running MCP server (ck --serve) for structured results, pagination, and streaming.
Support inline navigation (open file + jump to line/col).
Provide visual cues for index health and mode.
Expose command palette actions (Search, Search Selection, Reindex).
Deliver hybrid, semantic, or regex modes without breaking flow.
Make it Cursor-ready — leveraging MCP for richer, multi-tool integration.
Non-Goals (v1)
No team-shared search history or cloud index management.
No embedded LLM summaries or code rewriting.
No custom model tuning or embedding configuration via the extension (CLI handles that).
No multi-repo federation beyond the workspace root.
3. Target Users & Use Cases
Primary User:
Professional developer who prefers local tools over SaaS.
Works in large codebases with complex naming, needing both keyword and meaning search.
Already comfortable with rg, fzf, or ck CLI.
Secondary Users:
Cursor users who want an MCP-native experience for local or offline search.
OSS maintainers who want to quickly explore unfamiliar repos.
Core Use Cases:
“Find by meaning”
Developer types: “http server startup” → sees function calls like start_listener() or spawn_server().
“Search selection”
Highlights a function → runs ck semantic_search for similar patterns.
“TUI-like browsing”
Arrows navigate results, Enter jumps to code, Esc focuses query input.
“Instant reindex”
Runs ck reindex without leaving the editor; shows progress/status.
“Mode switch”
Quickly toggle between semantic, hybrid, and regex searches.
“Cursor integration”
MCP mode exposes structured search results for toolchains like Claude or Cursor’s code actions.
4. User Experience
4.1 Sidebar Layout
Top bar:
Input box for queries
Mode selector (Hybrid / Semantic / Regex)
Index status pill (green/yellow/red)
Body:
Scrollable list of results:
Each item: path:line + highlighted snippet
Click → opens in editor and jumps to location
Footer:
Toggle for showing raw CLI output (for power users)
4.2 Keyboard Interaction
Action	Keybinding	Description
Focus search bar	Ctrl+Shift+;	Opens sidebar and focuses query input
Search selection	Ctrl+Shift+'	Runs ck search on selected text
Navigate results	↑ / ↓	Move selection
Open selected result	Enter	Jump to file+line
Reindex	Command Palette (ck: Reindex)	Rebuilds index
4.3 Result Display
Results show filename, line number, and snippet.
Clicking a result:
Opens file.
Scrolls to line.
Briefly highlights the match.
4.4 Visual Style
Minimalist, terminal-inspired (monospaced, compact).
Uses VS Code theme colors.
Feels like a modern TUI: keyboard-centric, dense information, quick transitions.
5. Functional Requirements
5.1 Integration Modes
Mode	Mechanism	Description
CLI	Spawns ck binary with flags	Uses stdout/stderr to parse results
MCP	Starts or connects to ck --serve	Uses JSON-RPC to call hybrid_search, semantic_search, index_status, etc.
5.2 Commands
ck.search — Open sidebar and focus query input.
ck.searchSelection — Search current selection semantically.
ck.reindex — Trigger reindexing.
(Optional v1.1) ck.peekResults — Open “peek” view of matches inline.
5.3 Config Options
Setting	Type	Default	Description
ck.mode	string	cli	CLI or MCP mode
ck.cliPath	string	ck	Path to binary
ck.mcp.command	string	ck	MCP command
ck.mcp.args	array	[\"--serve\"]	Arguments for MCP server
ck.index.root	string	${workspaceFolder}	Root folder for indexing
ck.hybrid	boolean	true	Use hybrid search
ck.pageSize	number	50	Number of results per page
6. Technical Requirements
6.1 Language & Framework
Language: TypeScript
Target: VS Code extension API (>=1.93.0)
Frontend: WebviewViewProvider
Backend: Node.js child process + optional MCP client
6.2 Dependencies
jsonrpc-lite for MCP communication
typescript, @types/node, vsce for packaging
6.3 Performance
Search results should begin rendering <300ms after CLI output or MCP response.
No blocking UI on large outputs.
MCP connection should reconnect automatically on extension reload.
6.4 Security
Webview CSP: default-src 'none'; script-src 'nonce-*'; style-src 'unsafe-inline'.
No remote network calls.
Only interacts with local filesystem paths under workspace root.
7. Future Enhancements
Feature	Description	Priority
Streaming output	Show results as they’re emitted	High
Index health indicator	Call index_status periodically	High
Peek results	Inline “peek view” for matches	Medium
History + Re-run	Save last 10 queries	Medium
Workspace multi-root	Handle multiple ck roots	Medium
Result filters	File type / repo / path regex	Medium
Syntax highlighting	Render snippets with code coloration	Low
MCP discovery	Auto-detect running MCP server	Low
8. Success Metrics
Metric	Goal
Median search latency (local)	<500ms
Search results relevance satisfaction (user survey)	>80% positive
Adoption rate among existing ck users	50% of CLI users install extension
Cursor users connecting via MCP	≥25% of extension installs
Crash or hang rate	<1% of sessions
9. Release Plan
Phase 1: MVP (Weeks 1–3)
CLI mode search + sidebar UI.
Open-to-line navigation.
Command palette shortcuts.
Phase 2: MCP integration (Weeks 4–6)
JSON-RPC client integration.
Support hybrid_search and index_status.
Phase 3: UX polish (Weeks 7–8)
Add keyboard navigation.
Mode toggle + index status indicator.
Release v1.0 to Marketplace.
Phase 4: Cursor support (Weeks 9–10)
Register extension as MCP client.
Validate with Cursor team.
10. Competitive Landscape
Tool	Comparison
ripgrep / fzf	Fast keyword search, no semantic depth.
Sourcegraph / Cody	Semantic search via cloud, not local/offline.
Cursor MCP search	Great integration but not local or user-controlled.
ck (CLI)	Local, powerful, but CLI-only UX.
ck VS Code Extension	Merges ck’s power with editor-native discoverability.
11. Risks & Mitigation
Risk	Mitigation
MCP protocol changes	Keep versioned tool discovery & fallback to CLI mode.
Parsing CLI output	Standardize --json output mode for ck.
Large index performance	Paginate + stream results incrementally.
VS Code API changes	Use stable WebviewViewProvider interface.
12. Summary
This extension aims to bring ck’s semantic, hybrid, and regex power to the developer’s fingertips — directly inside the editor — with zero setup friction.
It’s fast, local, and pairs beautifully with ck’s philosophy: search code the way you think about it, not the way it’s written.


--- ck-vscode/README.md ---
# ck - Semantic Code Search for VS Code & Cursor

Brings the power of `ck` semantic code search directly into Visual Studio Code and Cursor.

## Features

### Search Capabilities
- **Semantic Search**: Find code by meaning, not just keywords
- **Hybrid Search** (Default): Combine semantic understanding with keyword precision
- **Regex Search**: Traditional pattern matching when you need it
- **Automatic Reranking**: Cross-encoder reranking enabled by default for best relevance (⚡ RERANK badge)
- **Smart Context**: 2 lines of context before/after matches for better understanding

### UI & UX
- **Crisp, Clean Interface**: Terminal-inspired monospace design
- **Live Results**: Search as you type with 300ms debouncing (TUI-style)
- **Visual Score Indicators**: Color-coded relevance scores (cyan/blue/yellow/orange) with visual bars
- **Line-by-Line Previews**: Context-aware preview with accurate line numbering
- **Relative Paths**: Clean file paths relative to workspace root
- **Keyboard Navigation**: ↑/↓ to navigate, Enter to open, Esc to reset
- **Match Highlighting**: Visual distinction for matching lines in previews

### Integration
- **Direct File Navigation**: Click any result to jump to exact location with brief highlight
- **Real-time Index Status**: Green dot when indexed, yellow when needs indexing
- **One-Click Reindexing**: Rebuild workspace index with progress notifications
- **Editor Integration**: "Search Selection" command for quick searches

## Requirements

- [ck](https://github.com/BeaconBay/ck) must be installed and available in your PATH
- Install with: `cargo install ck-search`

## Usage

### Commands

- `ck: Search` (`Ctrl+Shift+;` / `Cmd+Shift+;`) - Open search panel
- `ck: Search Selection` (`Ctrl+Shift+'` / `Cmd+Shift+'`) - Search selected text
- `ck: Reindex` - Force rebuild of search index

### Search Modes

- **Hybrid** - Combines semantic and keyword search (default)
- **Semantic** - Find code by concept and meaning
- **Regex** - Traditional grep-style pattern matching

### Keyboard Navigation

- `↑/↓` - Navigate results
- `Enter` - Open selected result or trigger search
- `Esc` - Return focus to search input

## Extension Settings

- `ck.mode` - Integration mode: `cli` (default) or `mcp`
- `ck.cliPath` - Path to ck binary (default: `ck`)
- `ck.defaultMode` - Default search mode: `hybrid`, `semantic`, or `regex`
- `ck.topK` - Maximum number of results (default: 100)
- `ck.threshold` - Minimum relevance threshold (default: 0.02)
- `ck.pageSize` - Results per page (default: 50)

## Installation

### For Cursor

```bash
cd ck-vscode
./install-cursor.sh
```

Then restart Cursor.

### For VS Code

```bash
cd ck-vscode
npm install
npm run compile
code --install-extension . --force
```

## Development

### Building

```bash
cd ck-vscode
npm install
npm run compile
```

### Testing

1. Open in VS Code
2. Press F5 to launch Extension Development Host
3. Test search functionality

### Packaging

```bash
npm run package
```

This creates a `.vsix` file you can install locally or publish to the marketplace.

## Roadmap

- [x] Phase 1: CLI mode + sidebar UI
- [x] Automatic reranking for better relevance
- [x] Visual score indicators and crisp UI
- [x] Line numbers and match highlighting
- [x] Relative path display
- [ ] Phase 2: MCP server integration for persistent connections
- [ ] Phase 3: Full syntax highlighting in previews
- [ ] Phase 4: Streaming results for large codebases
- [ ] Phase 5: Multi-workspace support

## License

Same as ck - Apache 2.0 or MIT

## More Information

- [ck on GitHub](https://github.com/BeaconBay/ck)
- [Report Issues](https://github.com/BeaconBay/ck/issues)


## Links discovered
- [ck](https://github.com/BeaconBay/ck)
- [ck on GitHub](https://github.com/BeaconBay/ck)
- [Report Issues](https://github.com/BeaconBay/ck/issues)

--- ck-vscode/src/cliAdapter.ts ---
/**
 * CLI adapter for spawning ck binary and parsing results
 */

import { spawn } from 'child_process';
import * as fs from 'fs';
import * as pathModule from 'path';
import * as vscode from 'vscode';
import { SearchOptions, SearchResponse, SearchResult, IndexStatus, IndexProgressUpdate } from './types';

export class CkCliAdapter {
  private cliPath: string;
  private progressEmitter = new vscode.EventEmitter<IndexProgressUpdate>();

  public readonly onIndexProgress = this.progressEmitter.event;

  constructor(cliPath: string) {
    this.cliPath = cliPath;
  }

  /**
   * Execute a search using the ck CLI
   */
  async search(options: SearchOptions): Promise<SearchResponse> {
    const args = this.buildSearchArgs(options);

    return new Promise((resolve, reject) => {
      const results: SearchResult[] = [];
      const errors: string[] = [];

      const child = spawn(this.cliPath, args, {
        cwd: options.path,
        shell: false
      });

      let stdout = '';
      let stderr = '';

      child.stdout.on('data', (data) => {
        stdout += data.toString();
      });

      child.stderr.on('data', (data) => {
        stderr += data.toString();
      });

      child.on('close', (code) => {
        if (code !== 0 && code !== 1) {
          // Code 1 is acceptable (no matches found)
          reject(new Error(`ck exited with code ${code}: ${stderr}`));
          return;
        }

        try {
          // Parse JSONL output
          const lines = stdout.split('\n').filter(line => line.trim());

          for (const line of lines) {
            try {
              const result = this.parseJsonlResult(line);
              if (result) {
                // Apply client-side threshold filtering for semantic/hybrid searches
                if (options.mode === 'semantic' || options.mode === 'hybrid') {
                  if (result.score !== undefined && options.threshold !== undefined) {
                    if (result.score >= options.threshold) {
                      results.push(result);
                    }
                  } else {
                    // If no score or no threshold, include it
                    results.push(result);
                  }
                } else {
                  // For regex searches, include all results
                  results.push(result);
                }
              }
            } catch (e) {
              console.warn('Failed to parse result line:', line, e);
            }
          }

          resolve({
            results,
            count: results.length,
            totalCount: results.length,
            hasMore: false
          });
        } catch (e) {
          reject(e);
        }
      });

      child.on('error', (err) => {
        reject(new Error(`Failed to spawn ck: ${err.message}`));
      });
    });
  }

  /**
   * Get index status for a path
   */
  async getIndexStatus(path: string): Promise<IndexStatus> {
    return new Promise((resolve, reject) => {
      const child = spawn(this.cliPath, ['--status', path], {
        shell: false
      });

      let stdout = '';
      let stderr = '';

      child.stdout.on('data', (data) => {
        stdout += data.toString();
      });

      child.stderr.on('data', (data) => {
        stderr += data.toString();
      });

      child.on('close', (code) => {
        // Parse status output
        // This is a simplified version - actual parsing depends on ck's status output format
        const exists = !stdout.includes('No index found');

        const indexDir = pathModule.join(path, '.ck');
        let lastModified: number | undefined;
        try {
          const stats = fs.statSync(indexDir);
          lastModified = Math.floor(stats.mtimeMs / 1000);
        } catch {
          // ignore missing directory or access issues
        }

        resolve({
          exists,
          path,
          totalFiles: this.extractNumber(stdout, /(\d+)\s+files/),
          totalChunks: this.extractNumber(stdout, /(\d+)\s+chunks/),
          indexPath: indexDir,
          lastModified
        });
      });

      child.on('error', (err) => {
        reject(new Error(`Failed to get index status: ${err.message}`));
      });
    });
  }

  async getDefaultCkignoreContent(indexRoot: string): Promise<string> {
    return new Promise((resolve, reject) => {
      const child = spawn(this.cliPath, ['--print-default-ckignore'], {
        cwd: indexRoot,
        shell: false
      });

      let stdout = '';
      let stderr = '';

      child.stdout.on('data', (data) => {
        stdout += data.toString();
      });

      child.stderr.on('data', (data) => {
        stderr += data.toString();
      });

      child.on('close', (code) => {
        if (code === 0) {
          resolve(stdout);
        } else {
          reject(new Error(`Failed to fetch default .ckignore (exit code ${code}): ${stderr}`));
        }
      });

      child.on('error', (err) => {
        reject(new Error(`Failed to spawn ck: ${err.message}`));
      });
    });
  }

  /**
   * Trigger reindexing
   */
  async reindex(path: string, progress?: vscode.Progress<{ message?: string; increment?: number }>): Promise<void> {
    return new Promise((resolve, reject) => {
      const child = spawn(this.cliPath, ['--index', path], {
        shell: false
      });

      let stderr = '';

      child.stderr.on('data', (data) => {
        const message = data.toString();
        stderr += message;

        // Report progress if available
        if (progress) {
          progress.report({ message: message.trim() });
        }

        const trimmed = message.trim();
        if (trimmed) {
          this.progressEmitter.fire({
            message: trimmed,
            source: 'cli',
            timestamp: Date.now()
          });
        }
      });

      child.on('close', (code) => {
        if (code === 0) {
          this.progressEmitter.fire({
            message: 'Reindexing complete',
            source: 'cli',
            timestamp: Date.now()
          });
          resolve();
        } else {
          this.progressEmitter.fire({
            message: `Reindexing failed (code ${code})`,
            source: 'cli',
            timestamp: Date.now()
          });
          reject(new Error(`Reindexing failed with code ${code}: ${stderr}`));
        }
      });

      child.on('error', (err) => {
        this.progressEmitter.fire({
          message: `Failed to reindex: ${err.message}`,
          source: 'cli',
          timestamp: Date.now()
        });
        reject(new Error(`Failed to reindex: ${err.message}`));
      });
    });
  }

  /**
   * Build command-line arguments for search
   */
  private buildSearchArgs(options: SearchOptions): string[] {
    const args: string[] = ['--jsonl'];

    // Search mode
    switch (options.mode) {
      case 'semantic':
        args.push('--sem');
        break;
      case 'hybrid':
        args.push('--hybrid');
        break;
      case 'regex':
        // Default mode, no flag needed
        break;
    }

    const shouldRerank =
      (options.mode === 'semantic' || options.mode === 'hybrid') && (options.rerank ?? true);
    if (shouldRerank) {
      args.push('--rerank');
    }

    if (options.rerankModel) {
      args.push('--rerank-model', options.rerankModel);
    }

    // Always show scores for semantic/hybrid
    if (options.mode === 'semantic' || options.mode === 'hybrid') {
      args.push('--scores');
    }

    // Always show line numbers
    args.push('-n');

    if (options.includeSnippet === false) {
      args.push('--no-snippet');
    }

    if (options.respectGitignore === false) {
      args.push('--no-ignore');
    }

    if (options.useDefaultExcludes === false) {
      args.push('--no-default-excludes');
    }

    // Query
    args.push(options.query);

    // File targets (include patterns)
    if (options.includePatterns && options.includePatterns.length > 0) {
      options.includePatterns
        .filter(pattern => pattern.trim().length > 0)
        .forEach(pattern => args.push(pattern.trim()));
    } else {
      // Default to searching the current workspace directory
      args.push('.');
    }

    // Optional parameters
    if (options.topK !== undefined) {
      args.push('--topk', options.topK.toString());
    }

    if (options.threshold !== undefined) {
      args.push('--threshold', options.threshold.toString());
    }

    if (options.caseInsensitive) {
      args.push('-i');
    }

    // Default to 2 lines of context for better snippets
    const contextLines = options.contextLines !== undefined ? options.contextLines : 2;
    if (contextLines > 0) {
      args.push('-C', contextLines.toString());
    }

    // Exclude patterns
    if (options.excludePatterns && options.excludePatterns.length > 0) {
      options.excludePatterns.forEach(pattern => {
        if (pattern.trim()) {
          args.push('--exclude', pattern.trim());
        }
      });
    }

    return args;
  }

  /**
   * Parse a JSONL result line from ck output
   */
  private parseJsonlResult(line: string): SearchResult | null {
    if (!line.trim()) {
      return null;
    }

    const data = JSON.parse(line);

    return {
      file: data.file || data.path,
      lineStart: data.line_start || data.span?.line_start || 1,
      lineEnd: data.line_end || data.span?.line_end || 1,
      byteStart: data.byte_start || data.span?.byte_start || 0,
      byteEnd: data.byte_end || data.span?.byte_end || 0,
      preview: data.snippet || data.preview || data.content || '',
      score: data.score,
      language: data.language || data.lang
    };
  }

  /**
   * Extract a number from text using regex
   */
  private extractNumber(text: string, regex: RegExp): number | undefined {
    const match = text.match(regex);
    return match ? parseInt(match[1], 10) : undefined;
  }

  dispose(): void {
    // CLI adapter does not maintain persistent resources
    this.progressEmitter.dispose();
  }
}


--- ck-vscode/src/extension.ts ---
/**
 * ck VS Code Extension
 *
 * Brings semantic code search powered by ck directly into VS Code
 */

import * as vscode from 'vscode';
import { CkSearchPanel } from './searchPanel';
import { CkConfig } from './types';

let searchPanel: CkSearchPanel | undefined;

export function activate(context: vscode.ExtensionContext) {
  console.log('ck extension is now active');

  // Load configuration
  const config = loadConfig();

  // Register the webview view provider
  const provider = new CkSearchPanel(context.extensionUri, config);
  searchPanel = provider;

  context.subscriptions.push(
    vscode.window.registerWebviewViewProvider(
      CkSearchPanel.viewType,
      provider,
      {
        webviewOptions: {
          retainContextWhenHidden: true
        }
      }
    )
  );

  // Register commands
  context.subscriptions.push(
    vscode.commands.registerCommand('ck.search', async () => {
      // Focus the search view
      await vscode.commands.executeCommand('ck.searchView.focus');
    })
  );

  context.subscriptions.push(
    vscode.commands.registerCommand('ck.searchSelection', async () => {
      const editor = vscode.window.activeTextEditor;
      if (!editor) {
        vscode.window.showWarningMessage('No active editor');
        return;
      }

      const selection = editor.selection;
      const text = editor.document.getText(selection);

      if (!text) {
        vscode.window.showWarningMessage('No text selected');
        return;
      }

      // Focus the search view and perform search
      await vscode.commands.executeCommand('ck.searchView.focus');

      if (searchPanel) {
        await searchPanel.search(text, config.defaultMode);
      }
    })
  );

  context.subscriptions.push(
    vscode.commands.registerCommand('ck.reindex', async () => {
      if (searchPanel) {
        await searchPanel.reindex();
      }
    })
  );

  // Listen for configuration changes
  context.subscriptions.push(
    vscode.workspace.onDidChangeConfiguration((e) => {
      if (e.affectsConfiguration('ck')) {
        const newConfig = loadConfig();
        if (searchPanel) {
          searchPanel.updateConfig(newConfig);
        }
      }
    })
  );

  // Check if ck binary is available
  checkCkAvailability(config.cliPath);
}

export function deactivate() {
  if (searchPanel) {
    searchPanel.dispose();
    searchPanel = undefined;
  }
}

/**
 * Load configuration from VS Code settings
 */
function loadConfig(): CkConfig {
  const config = vscode.workspace.getConfiguration('ck');

  return {
    mode: config.get('mode', 'cli') as 'cli' | 'mcp',
    cliPath: config.get('cliPath', 'ck'),
    mcpCommand: config.get('mcp.command', 'ck'),
    mcpArgs: config.get('mcp.args', ['--serve']),
    indexRoot: config.get('index.root', '${workspaceFolder}'),
    defaultMode: config.get('defaultMode', 'semantic') as 'hybrid' | 'semantic' | 'regex',
    pageSize: config.get('pageSize', 25),
    threshold: config.get('threshold', 0.6),
    topK: config.get('topK', 50),
    contextLines: config.get('contextLines', 4)
  };
}

/**
 * Check if ck binary is available on PATH
 */
async function checkCkAvailability(cliPath: string) {
  const { spawn } = require('child_process');

  const child = spawn(cliPath, ['--version'], { shell: false });

  let available = false;

  child.on('close', (code: number) => {
    if (code === 0) {
      available = true;
    } else {
      vscode.window.showWarningMessage(
        `ck binary not found at "${cliPath}". Please install ck or update the "ck.cliPath" setting.`,
        'Install ck',
        'Open Settings'
      ).then((selection) => {
        if (selection === 'Install ck') {
          vscode.env.openExternal(vscode.Uri.parse('https://github.com/BeaconBay/ck#installation'));
        } else if (selection === 'Open Settings') {
          vscode.commands.executeCommand('workbench.action.openSettings', 'ck.cliPath');
        }
      });
    }
  });

  child.on('error', () => {
    vscode.window.showWarningMessage(
      `ck binary not found at "${cliPath}". Please install ck or update the "ck.cliPath" setting.`,
      'Install ck',
      'Open Settings'
    ).then((selection) => {
      if (selection === 'Install ck') {
        vscode.env.openExternal(vscode.Uri.parse('https://github.com/BeaconBay/ck#installation'));
      } else if (selection === 'Open Settings') {
        vscode.commands.executeCommand('workbench.action.openSettings', 'ck.cliPath');
      }
    });
  });
}


--- ck-vscode/webview/main.js ---
/**
 * Webview script for ck search panel - Crisp UI
 * Handles UI interactions and communication with extension
 */

(function () {
  const vscode = acquireVsCodeApi();

  // State
  let currentResults = [];
  let selectedIndex = -1;
  let config = null;
  let searchTimeout = null;
  let currentMode = 'semantic';
  let currentQuery = '';
  let currentQueryTokens = [];

  // DOM elements
  const searchInput = document.getElementById('searchInput');
  const modeSelector = document.getElementById('modeSelector');
  const reindexButton = document.getElementById('reindexButton');
  const includeInput = document.getElementById('includeInput');
  const excludeInput = document.getElementById('excludeInput');
  const indexStatus = document.getElementById('indexStatus');
  const indexProgress = document.getElementById('indexProgress');
  const indexProgressText = indexProgress ? indexProgress.querySelector('.progress-text') : null;
  const ckignoreButton = document.getElementById('ckignoreButton');
  const refreshStatusButton = document.getElementById('refreshStatusButton');
  const loadingIndicator = document.getElementById('loadingIndicator');
  const resultsContainer = document.getElementById('resultsContainer');
  const errorContainer = document.getElementById('errorContainer');
  const resultCount = document.getElementById('resultCount');
  let hideProgressTimeout = null;

  // Event listeners
  searchInput.addEventListener('input', handleSearchInput);
  searchInput.addEventListener('keydown', handleSearchKeydown);
  modeSelector.addEventListener('change', handleModeChange);
  reindexButton.addEventListener('click', handleReindexClick);
  if (ckignoreButton) {
    ckignoreButton.addEventListener('click', () => {
      vscode.postMessage({ type: 'openCkignore' });
    });
  }
  if (refreshStatusButton) {
    refreshStatusButton.addEventListener('click', () => {
      vscode.postMessage({ type: 'getIndexStatus' });
    });
  }

  // Trigger search when include/exclude patterns change
  if (includeInput) {
    includeInput.addEventListener('input', () => {
      const query = searchInput.value.trim();
      if (query) {
        clearTimeout(searchTimeout);
        searchTimeout = setTimeout(() => {
          performSearch(query, modeSelector.value);
        }, 300);
      }
    });
  }

  if (excludeInput) {
    excludeInput.addEventListener('input', () => {
      const query = searchInput.value.trim();
      if (query) {
        clearTimeout(searchTimeout);
        searchTimeout = setTimeout(() => {
          performSearch(query, modeSelector.value);
        }, 300);
      }
    });
  }

  // Handle messages from extension
  window.addEventListener('message', (event) => {
    const message = event.data;

    switch (message.type) {
      case 'config':
        handleConfig(message.config);
        break;
      case 'searchStarted':
        handleSearchStarted(message.query, message.mode);
        break;
      case 'searchResults':
        handleSearchResults(message);
        break;
      case 'searchError':
        handleSearchError(message.error);
      break;
    case 'indexStatus':
      handleIndexStatus(message.status);
      break;
    case 'indexProgress':
      handleIndexProgress(message.update);
      break;
    }
  });

  // Request initial index status
  vscode.postMessage({ type: 'getIndexStatus' });

  /**
   * Handle search input with debouncing
   */
  function handleSearchInput() {
    clearTimeout(searchTimeout);

    const query = searchInput.value.trim();
    if (!query) {
      showEmptyState();
      return;
    }

    // Debounce search (300ms like TUI)
    searchTimeout = setTimeout(() => {
      performSearch(query, modeSelector.value);
    }, 300);
  }

  /**
   * Handle keyboard navigation in search input
   */
  function handleSearchKeydown(e) {
    if (e.key === 'ArrowDown') {
      e.preventDefault();
      selectNext();
    } else if (e.key === 'ArrowUp') {
      e.preventDefault();
      selectPrevious();
    } else if (e.key === 'Enter') {
      e.preventDefault();
      if (selectedIndex >= 0 && selectedIndex < currentResults.length) {
        openResult(currentResults[selectedIndex]);
      } else {
        performSearch(searchInput.value.trim(), modeSelector.value);
      }
    } else if (e.key === 'Escape') {
      selectedIndex = -1;
      updateSelection();
      searchInput.focus();
    }
  }

  /**
   * Handle mode selector change
   */
  function handleModeChange() {
    currentMode = modeSelector.value;
    const query = searchInput.value.trim();
    if (query) {
      performSearch(query, currentMode);
    }
  }

  /**
   * Handle reindex button click
   */
  function handleReindexClick() {
    vscode.postMessage({ type: 'reindex' });
  }

  /**
   * Perform search
   */
  function performSearch(query, mode) {
    currentMode = mode;
    currentQuery = query;
    currentQueryTokens = buildQueryTokens(query);

    // Parse include/exclude patterns (comma or space separated)
    const includePatterns = includeInput && includeInput.value.trim()
      ? includeInput.value.split(/[,\s]+/).map(p => p.trim()).filter(p => p)
      : [];

    const excludePatterns = excludeInput && excludeInput.value.trim()
      ? excludeInput.value.split(/[,\s]+/).map(p => p.trim()).filter(p => p)
      : [];

    vscode.postMessage({
      type: 'search',
      query,
      mode,
      includePatterns: includePatterns.length > 0 ? includePatterns : undefined,
      excludePatterns: excludePatterns.length > 0 ? excludePatterns : undefined
    });
  }

  /**
   * Handle config from extension
   */
  function handleConfig(newConfig) {
    config = newConfig;
    currentMode = config.defaultMode;
    modeSelector.value = currentMode;
  }

  /**
   * Handle search started
   */
  function handleSearchStarted(query, mode) {
    currentMode = mode;
    loadingIndicator.classList.remove('hidden');
    errorContainer.classList.add('hidden');
    resultsContainer.innerHTML = '';
    currentResults = [];
    selectedIndex = -1;
  }

  /**
   * Handle search results
   */
  function handleSearchResults(message) {
    loadingIndicator.classList.add('hidden');
    currentResults = message.results || [];

    if (currentResults.length === 0) {
      showNoResults();
    } else {
      renderResults(currentResults);
    }

    updateResultCount(message.count, message.totalCount, message.hasMore);
  }

  /**
   * Handle search error
   */
  function handleSearchError(error) {
    loadingIndicator.classList.add('hidden');
    errorContainer.classList.remove('hidden');
    errorContainer.textContent = `Error: ${error}`;
    currentResults = [];
  }

  /**
   * Handle index status
   */
  function handleIndexStatus(status) {
    if (!indexStatus) {
      return;
    }

    const indicator = indexStatus.querySelector('.status-indicator');
    const text = indexStatus.querySelector('.status-text');

    if (!status) {
      indicator?.classList.remove('healthy', 'warning', 'error');
      if (text) {
        text.textContent = 'Index status unavailable';
      }
      return;
    }

    indexStatus.dataset.path = status.path || '';
    indexStatus.title = 'ck respects .gitignore, .ckignore, include/exclude filters, and default VS Code excludes';

    const parts = [];
    const fileCount = typeof status.totalFiles === 'number'
      ? status.totalFiles
      : typeof status.estimatedFileCount === 'number'
        ? status.estimatedFileCount
        : undefined;

    if (status.exists) {
      indicator?.classList.add('healthy');
      indicator?.classList.remove('warning', 'error');

      if (typeof fileCount === 'number') {
        const approxPrefix = status.totalFiles === undefined && status.estimatedFileCount !== undefined ? '≈' : '';
        parts.push(`${approxPrefix}${fileCount.toLocaleString()} ${fileCount === 1 ? 'file' : 'files'}`);
      }

      if (typeof status.totalChunks === 'number') {
        parts.push(`${status.totalChunks.toLocaleString()} chunks`);
      }

      if (typeof status.indexSizeBytes === 'number') {
        parts.push(formatBytes(status.indexSizeBytes));
      }

      if (typeof status.lastModified === 'number' && status.lastModified > 0) {
        parts.push(`updated ${formatRelativeTime(status.lastModified)}`);
      }

      if (parts.length === 0) {
        parts.push('Ready');
      }
    } else {
      indicator?.classList.add('warning');
      indicator?.classList.remove('healthy', 'error');
      parts.push('Not indexed');
      if (typeof fileCount === 'number') {
        parts.push(`~${fileCount.toLocaleString()} files in scope`);
      }
    }

    if (text) {
      text.textContent = parts.join(' · ');
    }
  }

  function handleIndexProgress(update) {
    if (!indexProgress || !indexProgressText || !update) {
      return;
    }

    const message = typeof update.message === 'string' ? update.message.trim() : '';
    if (!message) {
      return;
    }

    clearTimeout(hideProgressTimeout);

    let displayMessage = message;
    if (typeof update.total === 'number' && typeof update.progress === 'number' && update.total > 0) {
      const current = Math.min(update.progress, update.total);
      displayMessage = `${message} (${current}/${update.total})`;
    }

    indexProgress.classList.remove('hidden');
    indexProgress.dataset.source = update.source || '';
    indexProgressText.textContent = displayMessage;

    const lowered = message.toLowerCase();
    const completed = ['complete', 'completed', 'success', 'done', 'indexed'].some((token) => lowered.includes(token));

    const hideDelay = completed ? 3000 : 9000;
    hideProgressTimeout = setTimeout(() => {
      indexProgress.classList.add('hidden');
    }, hideDelay);

    if (completed) {
      setTimeout(() => {
        vscode.postMessage({ type: 'getIndexStatus' });
      }, 500);
    }
  }

  /**
   * Render search results grouped by file with enhanced formatting
   */
  function renderResults(results) {
    resultsContainer.innerHTML = '';

    const partitions = partitionResultsByThreshold(results);
    if (partitions.primary.length === 0 && partitions.secondary.length === 0) {
      showNoResults();
      return;
    }

    const toolbar = createResultsToolbar(partitions.secondary.length > 0 && partitions.threshold > 0);
    resultsContainer.appendChild(toolbar);

    const primaryWrapper = document.createElement('div');
    primaryWrapper.className = 'file-group-stack';
    resultsContainer.appendChild(primaryWrapper);

    const primaryGroups = groupResultsByFile(partitions.primary);
    primaryGroups.forEach((items, filePath) => {
      primaryWrapper.appendChild(createFileGroup(filePath, items, false, 'primary'));
    });

    if (partitions.secondary.length > 0 && partitions.threshold > 0) {
      const nearHeader = document.createElement('div');
      nearHeader.className = 'near-matches-header';
      nearHeader.textContent = `Near matches (score < ${partitions.threshold.toFixed(2)})`;
      resultsContainer.appendChild(nearHeader);

      const secondaryWrapper = document.createElement('div');
      secondaryWrapper.className = 'file-group-stack near-matches';
      resultsContainer.appendChild(secondaryWrapper);

      const secondaryGroups = groupResultsByFile(partitions.secondary);
      secondaryGroups.forEach((items, filePath) => {
        secondaryWrapper.appendChild(createFileGroup(filePath, items, true, 'secondary'));
      });
    }
  }

  /**
   * Build a collapsible group of results for a single file
   */
  function createFileGroup(filePath, items, collapsed = false, variant = 'primary') {
    const group = document.createElement('div');
    group.className = 'file-group';
    if (variant === 'secondary') {
      group.classList.add('file-group--secondary');
    }

    const header = document.createElement('div');
    header.className = 'file-group-header';

    const toggleIcon = document.createElement('span');
    toggleIcon.className = 'toggle-icon';
    toggleIcon.textContent = '▾';

    const nameWrapper = document.createElement('div');
    nameWrapper.className = 'file-group-title';

    const normalizedPath = filePath.replace(/\\/g, '/');
    const baseName = getBasename(normalizedPath);
    const parentPath = normalizedPath.slice(0, normalizedPath.length - baseName.length).replace(/\/$/, '');

    const baseNameSpan = document.createElement('span');
    baseNameSpan.className = 'file-group-basename';
    baseNameSpan.textContent = baseName;
    nameWrapper.appendChild(baseNameSpan);

    if (parentPath) {
      const parentSpan = document.createElement('span');
      parentSpan.className = 'file-group-subpath';
      parentSpan.textContent = ` · ${parentPath}`;
      nameWrapper.appendChild(parentSpan);
    }

    nameWrapper.title = items[0].result.absolutePath || filePath;

    const metricsWrapper = document.createElement('div');
    metricsWrapper.className = 'file-group-metrics';

    const matchCountPill = document.createElement('span');
    matchCountPill.className = 'metric-pill metric-pill--count';
    matchCountPill.textContent = items.length.toString();
    metricsWrapper.appendChild(matchCountPill);

    const bestScore = items
      .map(({ result }) => (typeof result.score === 'number' ? result.score : undefined))
      .filter((score) => score !== undefined);
    if (bestScore.length > 0) {
      const topScore = Math.max(...bestScore);
      const scorePill = document.createElement('span');
      scorePill.className = `metric-pill metric-pill--score ${getScoreLabelClass(topScore)}`;
      scorePill.textContent = topScore.toFixed(topScore >= 0.1 ? 2 : 3);
      metricsWrapper.appendChild(scorePill);
    }

    header.appendChild(toggleIcon);
    header.appendChild(nameWrapper);
    header.appendChild(metricsWrapper);

    const resultsDiv = document.createElement('div');
    resultsDiv.className = 'file-group-results';

    items.forEach(({ result, index }) => {
      const item = createResultItem(result, index, true, variant);
      resultsDiv.appendChild(item);
    });

    header.addEventListener('click', (event) => {
      event.preventDefault();
      const shouldCollapse = !group.classList.contains('collapsed');
      setGroupCollapsed(group, shouldCollapse);
    });

    header.addEventListener('dblclick', (event) => {
      event.preventDefault();
      event.stopPropagation();
      setGroupCollapsed(group, false);
      if (items.length > 0) {
        openResult(items[0].result);
      }
    });

    group.appendChild(header);
    group.appendChild(resultsDiv);
    setGroupCollapsed(group, collapsed);

    return group;
  }

  /**
   * Create a result item element with improved snippet formatting
   */
  function createResultItem(result, index, inGroup = false, variant = 'primary') {
    const item = document.createElement('div');
    item.className = 'result-item';
    if (variant === 'secondary') {
      item.classList.add('result-item--secondary');
    }
    item.dataset.index = index;

    const header = document.createElement('div');
    header.className = 'result-header';

    if (!inGroup) {
      const file = document.createElement('div');
      file.className = 'result-file';
      file.textContent = result.file;
      file.title = result.absolutePath || result.file;
      header.appendChild(file);
    }

    const location = document.createElement('div');
    location.className = 'result-location';

    const lineStart = result.lineStart ?? 0;
    const lineEnd = result.lineEnd ?? result.lineStart;
    const rangeLabel = lineEnd && lineEnd !== lineStart ? `L${lineStart}–${lineEnd}` : `L${lineStart}`;

    const lineNum = document.createElement('span');
    lineNum.className = 'result-line';
    lineNum.textContent = rangeLabel;
    location.appendChild(lineNum);

    if (typeof result.score === 'number') {
      const scoreLabel = document.createElement('span');
      scoreLabel.className = `result-score-label ${getScoreLabelClass(result.score)}`;
      scoreLabel.textContent = `(${result.score.toFixed(2)})`;
      location.appendChild(scoreLabel);
    }

    header.appendChild(location);
    item.appendChild(header);

    const preview = createPreview(result);
    item.appendChild(preview);

    item.addEventListener('click', () => {
      openResult(result);
    });

    return item;
  }

  /**
   * Create preview section with syntax-aware highlighting
   */
  function createPreview(result) {
    const preview = document.createElement('div');
    preview.className = 'result-preview';

    const previewText = result.preview || '';
    const lines = previewText.split('\n');
    const startLine = result.lineStart ?? 0;
    const language = mapLanguage(result.language, result.file);
    const tokens = currentQueryTokens;

    lines.forEach((line, idx) => {
      const lineDiv = document.createElement('div');
      lineDiv.className = 'preview-line';

      const lineNumberValue = startLine + idx;
      lineDiv.addEventListener('click', (event) => {
        event.stopPropagation();
        openResultAtLine(result, lineNumberValue);
      });

      const lineNum = document.createElement('span');
      lineNum.className = 'line-number';
      lineNum.textContent = lineNumberValue.toString();
      lineDiv.appendChild(lineNum);

      const content = document.createElement('span');
      content.className = 'line-content';
      renderLineContent(content, line, language, tokens);
      lineDiv.appendChild(content);

      if (content.querySelector('.match-highlight')) {
        lineDiv.classList.add('match-line');
      }

      preview.appendChild(lineDiv);
    });

    return preview;
  }

  function createResultsToolbar(hasSecondaryGroups) {
    const toolbar = document.createElement('div');
    toolbar.className = 'results-toolbar';

    const label = document.createElement('span');
    label.className = 'toolbar-label';
    label.textContent = 'Search results';

    const buttons = document.createElement('div');
    buttons.className = 'toolbar-buttons';

    const expandBtn = document.createElement('button');
    expandBtn.className = 'toolbar-button';
    expandBtn.textContent = 'Expand all';
    expandBtn.addEventListener('click', () => setAllGroupsCollapsed(false));

    const collapseBtn = document.createElement('button');
    collapseBtn.className = 'toolbar-button';
    collapseBtn.textContent = 'Collapse all';
    collapseBtn.addEventListener('click', () => setAllGroupsCollapsed(true));

    buttons.appendChild(expandBtn);
    buttons.appendChild(collapseBtn);

    if (hasSecondaryGroups) {
      const collapseWeakBtn = document.createElement('button');
      collapseWeakBtn.className = 'toolbar-button';
      collapseWeakBtn.textContent = 'Collapse weak';
      collapseWeakBtn.addEventListener('click', () => {
        resultsContainer.querySelectorAll('.file-group--secondary').forEach((group) => {
          setGroupCollapsed(group, true);
        });
      });
      buttons.appendChild(collapseWeakBtn);
    }

    toolbar.appendChild(label);
    toolbar.appendChild(buttons);

    return toolbar;
  }

  function setAllGroupsCollapsed(collapsed) {
    resultsContainer.querySelectorAll('.file-group').forEach((group) => {
      setGroupCollapsed(group, collapsed);
    });
  }

  function setGroupCollapsed(group, collapsed) {
    if (collapsed) {
      group.classList.add('collapsed');
    } else {
      group.classList.remove('collapsed');
    }

    const icon = group.querySelector('.file-group-header .toggle-icon');
    if (icon) {
      icon.textContent = collapsed ? '▸' : '▾';
    }
  }

  function partitionResultsByThreshold(results) {
    const threshold = typeof config?.threshold === 'number' ? config.threshold : 0;
    const primary = [];
    const secondary = [];

    const hasScores = results.some((result) => typeof result.score === 'number');

    results.forEach((result, index) => {
      const entry = { result, index };
      if (hasScores && threshold > 0 && typeof result.score === 'number' && result.score < threshold) {
        secondary.push(entry);
      } else {
        primary.push(entry);
      }
    });

    return {
      primary,
      secondary,
      threshold: hasScores ? threshold : 0,
    };
  }

  function groupResultsByFile(entries) {
    const groups = new Map();
    entries.forEach((entry) => {
      const key = entry.result.file;
      if (!groups.has(key)) {
        groups.set(key, []);
      }
      groups.get(key).push(entry);
    });
    groups.forEach((groupEntries) => {
      groupEntries.sort((a, b) => {
        const lineA = a.result.lineStart ?? 0;
        const lineB = b.result.lineStart ?? 0;
        return lineA - lineB;
      });
    });
    return groups;
  }
  function getScoreLabelClass(score) {
    if (score >= 0.85) return 'score-label-excellent';
    if (score >= 0.65) return 'score-label-good';
    if (score >= 0.45) return 'score-label-fair';
    return 'score-label-poor';
  }

  function buildQueryTokens(query) {
    if (!query) {
      return [];
    }
    return Array.from(
      new Set(
        query
          .split(/[,\s]+/)
          .map((token) => token.trim())
          .filter((token) => token.length >= 2)
      )
    );
  }

  function mapLanguage(language, filePath) {
    const lang = (language || '').toLowerCase();
    if (lang) {
      return normalizeLanguageAlias(lang);
    }

    if (filePath) {
      const lower = filePath.toLowerCase();
      const extMatch = lower.match(/\.([a-z0-9]+)$/);
      if (extMatch) {
        return normalizeLanguageAlias(extMatch[1]);
      }
    }

    return 'plain';
  }

  function normalizeLanguageAlias(alias) {
    switch (alias) {
      case 'ts':
      case 'tsx':
      case 'typescript':
        return 'typescript';
      case 'js':
      case 'jsx':
      case 'javascript':
        return 'javascript';
      case 'py':
      case 'python':
        return 'python';
      case 'rs':
      case 'rust':
        return 'rust';
      case 'go':
      case 'golang':
        return 'go';
      case 'java':
        return 'java';
      case 'cs':
      case 'csharp':
        return 'csharp';
      case 'kt':
      case 'kotlin':
        return 'kotlin';
      case 'rb':
      case 'ruby':
        return 'ruby';
      case 'php':
        return 'php';
      case 'sh':
      case 'bash':
      case 'shell':
        return 'shell';
      case 'json':
        return 'json';
      case 'yaml':
      case 'yml':
        return 'yaml';
      default:
        return 'plain';
    }
  }

  function renderLineContent(container, line, language, tokens) {
    const escaped = escapeHtml(line);
    const syntaxHighlighted = applySyntaxHighlight(escaped, language);
    const temp = document.createElement('span');
    temp.innerHTML = syntaxHighlighted || '&nbsp;';
    if (tokens.length > 0) {
      applyTokenHighlights(temp, tokens);
    }
    container.innerHTML = temp.innerHTML || '&nbsp;';
  }

  function applySyntaxHighlight(html, language) {
    switch (language) {
      case 'javascript':
      case 'typescript':
      case 'java':
      case 'csharp':
      case 'kotlin':
      case 'php':
        return highlightCStyle(html);
      case 'rust':
        return highlightRust(html);
      case 'python':
        return highlightPython(html);
      case 'go':
        return highlightGo(html);
      case 'ruby':
        return highlightRuby(html);
      case 'shell':
        return highlightShell(html);
      case 'json':
        return highlightJson(html);
      case 'yaml':
        return highlightYaml(html);
      default:
        return html;
    }
  }

  function highlightCStyle(html) {
    html = highlightPattern(html, /(\/{2}[^<]*)$/m, 'comment');
    html = highlightPattern(html, /(\/\*[^*]*\*\/)/g, 'comment');
    html = highlightPattern(html, /(["'`])(?:\\.|(?!\1).)*\1/g, 'string');
    html = highlightPattern(html, /\b(0x[0-9a-fA-F]+|\d+(?:\.\d+)?)\b/g, 'number');
    html = highlightKeywords(html, [
      'const', 'let', 'var', 'function', 'return', 'if', 'else', 'for', 'while',
      'switch', 'case', 'break', 'continue', 'try', 'catch', 'finally', 'throw',
      'import', 'from', 'export', 'extends', 'implements', 'new', 'class',
      'interface', 'enum', 'public', 'private', 'protected', 'static', 'async',
      'await', 'yield', 'package', 'namespace', 'using'
    ]);
    html = highlightFunctionCalls(html);
    return html;
  }

  function highlightRust(html) {
    html = highlightPattern(html, /(\/\/[^<]*)$/m, 'comment');
    html = highlightPattern(html, /(["'])(?:\\.|(?!\1).)*\1/g, 'string');
    html = highlightPattern(html, /\b\d+(?:_\d+)*(?:\.\d+(?:_\d+)*)?\b/g, 'number');
    html = highlightKeywords(html, [
      'fn', 'let', 'mut', 'pub', 'struct', 'enum', 'impl', 'trait', 'match',
      'if', 'else', 'while', 'loop', 'for', 'use', 'mod', 'const', 'static',
      'ref', 'crate', 'super', 'Self', 'self', 'return', 'async', 'await'
    ]);
    html = highlightFunctionCalls(html);
    return html;
  }

  function highlightPython(html) {
    html = highlightPattern(html, /(#.*)$/m, 'comment');
    html = highlightPattern(html, /("""[\s\S]*?"""|'''[\s\S]*?'''|"(?:\\.|[^"])*"|'(?:\\.|[^'])*')/g, 'string');
    html = highlightPattern(html, /\b\d+(?:\.\d+)?\b/g, 'number');
    html = highlightKeywords(html, [
      'def', 'return', 'if', 'elif', 'else', 'for', 'while', 'import', 'from',
      'class', 'try', 'except', 'with', 'as', 'lambda', 'yield', 'pass', 'break',
      'continue', 'async', 'await', 'raise', 'True', 'False', 'None'
    ]);
    html = highlightFunctionCalls(html);
    return html;
  }

  function highlightGo(html) {
    html = highlightPattern(html, /(\/\/[^<]*)$/m, 'comment');
    html = highlightPattern(html, /(["'])(?:\\.|(?!\1).)*\1/g, 'string');
    html = highlightPattern(html, /\b\d+(?:\.\d+)?\b/g, 'number');
    html = highlightKeywords(html, [
      'func', 'var', 'const', 'return', 'if', 'else', 'for', 'range', 'import',
      'package', 'type', 'struct', 'interface', 'switch', 'case', 'default',
      'go', 'defer', 'map'
    ]);
    html = highlightFunctionCalls(html);
    return html;
  }

  function highlightRuby(html) {
    html = highlightPattern(html, /(#.*)$/m, 'comment');
    html = highlightPattern(html, /("(?:\\.|[^"])*"|'(?:\\.|[^'])*')/g, 'string');
    html = highlightPattern(html, /\b\d+(?:\.\d+)?\b/g, 'number');
    html = highlightKeywords(html, [
      'def', 'return', 'if', 'elsif', 'else', 'end', 'do', 'while', 'until',
      'class', 'module', 'include', 'extend', 'begin', 'rescue', 'ensure',
      'raise', 'yield', 'self', 'true', 'false', 'nil'
    ]);
    html = highlightFunctionCalls(html);
    return html;
  }

  function highlightShell(html) {
    html = highlightPattern(html, /(#.*)$/m, 'comment');
    html = highlightPattern(html, /("(?:\\.|[^"])*"|'(?:\\.|[^'])*')/g, 'string');
    html = highlightKeywords(html, [
      'if', 'then', 'else', 'elif', 'fi', 'for', 'while', 'do', 'done', 'in',
      'case', 'esac', 'function', 'return'
    ]);
    return html;
  }

  function highlightJson(html) {
    html = highlightPattern(html, /("(?:\\.|[^"])*")(?=\s*:)/g, 'keyword');
    html = highlightPattern(html, /("(?:\\.|[^"])*")/g, 'string');
    html = highlightPattern(html, /\b(true|false|null)\b/g, 'keyword');
    html = highlightPattern(html, /\b-?\d+(?:\.\d+)?\b/g, 'number');
    return html;
  }

  function highlightYaml(html) {
    html = highlightPattern(html, /(#.*)$/m, 'comment');
    html = highlightPattern(html, /("(?:\\.|[^"])*"|'(?:\\.|[^'])*')/g, 'string');
    html = highlightPattern(html, /^(\s*-)/gm, 'operator');
    html = highlightPattern(html, /(^|\s)([A-Za-z_\-][A-Za-z0-9_\-]*)(?=\s*:)/gm, 'keyword');
    return html;
  }

  function highlightPattern(html, regex, tokenClass) {
    return html.replace(regex, (match) => '<span class="token ' + tokenClass + '">' + match + '</span>');
  }

  function highlightKeywords(html, keywords) {
    const pattern = new RegExp('\\b(' + keywords.map(escapeRegExp).join('|') + ')\\b', 'g');
    return html.replace(pattern, '<span class="token keyword">$1</span>');
  }

  function highlightFunctionCalls(html) {
    return html.replace(/\b([A-Za-z_][A-Za-z0-9_]*)\s*(?=\()/g, '<span class="token function">$1</span>');
  }

  function applyTokenHighlights(root, tokens) {
    if (tokens.length === 0) {
      return;
    }
    const pattern = new RegExp('(' + tokens.map(escapeRegExp).join('|') + ')', 'gi');
    const walker = document.createTreeWalker(root, NodeFilter.SHOW_TEXT);
    const nodes = [];
    while (walker.nextNode()) {
      nodes.push(walker.currentNode);
    }
    nodes.forEach((node) => {
      const text = node.nodeValue;
      if (!text) {
        return;
      }
      pattern.lastIndex = 0;
      if (!pattern.test(text)) {
        return;
      }
      pattern.lastIndex = 0;
      const frag = document.createDocumentFragment();
      let lastIndex = 0;
      let match;
      while ((match = pattern.exec(text)) !== null) {
        const start = match.index;
        const end = start + match[0].length;
        if (start > lastIndex) {
          frag.appendChild(document.createTextNode(text.slice(lastIndex, start)));
        }
        const mark = document.createElement('mark');
        mark.className = 'match-highlight';
        mark.textContent = match[0];
        frag.appendChild(mark);
        lastIndex = end;
      }
      if (lastIndex < text.length) {
        frag.appendChild(document.createTextNode(text.slice(lastIndex)));
      }
      node.parentNode.replaceChild(frag, node);
    });
  }

  function escapeHtml(text) {
    return text
      .replace(/&/g, '&amp;')
      .replace(/</g, '&lt;')
      .replace(/>/g, '&gt;')
      .replace(/"/g, '&quot;')
      .replace(/'/g, '&#39;');
  }

  function escapeRegExp(text) {
    return text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
  }

  /**
   * Get basename from path
   */
  function getBasename(filePath) {
    const parts = filePath.split(/[/\\]/);
    return parts[parts.length - 1];
  }

  /**
   * Open a result in the editor
   */
  function openResult(result) {
    openResultAtLine(result, result.lineStart);
  }

  function openResultAtLine(result, lineNumber) {
    vscode.postMessage({
      type: 'openFile',
      file: result.absolutePath || result.file,
      line: Math.max(1, lineNumber || 1)
    });
  }

  /**
   * Navigate to next result
   */
  function selectNext() {
    if (currentResults.length === 0) return;

    selectedIndex = (selectedIndex + 1) % currentResults.length;
    updateSelection();
  }

  /**
   * Navigate to previous result
   */
  function selectPrevious() {
    if (currentResults.length === 0) return;

    selectedIndex = selectedIndex <= 0 ? currentResults.length - 1 : selectedIndex - 1;
    updateSelection();
  }

  /**
   * Update visual selection
   */
  function updateSelection() {
    const items = resultsContainer.querySelectorAll('.result-item');

    items.forEach((item, index) => {
      if (index === selectedIndex) {
        item.classList.add('selected');
        item.scrollIntoView({ block: 'nearest', behavior: 'smooth' });
      } else {
        item.classList.remove('selected');
      }
    });
  }

  /**
   * Show empty state
   */
  function showEmptyState() {
    resultsContainer.innerHTML = `
      <div class="empty-state">
        <p>Enter a search query to find code</p>
        <p class="hint">Semantic search understands meaning, not just keywords</p>
      </div>
    `;
    currentResults = [];
    resultCount.innerHTML = '';
  }

  /**
   * Show no results state
   */
  function showNoResults() {
    resultsContainer.innerHTML = `
      <div class="empty-state">
        <p>No results found</p>
        <p class="hint">Try a different query or search mode</p>
      </div>
    `;
    resultCount.innerHTML = '<span class="count-text">0 results</span>';
  }

  /**
   * Update result count display with rerank badge
   */
  function updateResultCount(count, totalCount, hasMore) {
    const countText = document.createElement('span');
    countText.className = 'count-text';

    let text = `${count} result${count !== 1 ? 's' : ''}`;
    if (totalCount && totalCount !== count) {
      text += ` of ${totalCount}`;
    }
    if (hasMore) {
      text += '+';
    }
    countText.textContent = text;

    // Show rerank badge for semantic/hybrid
    const showRerank = currentMode === 'semantic' || currentMode === 'hybrid';

    const html = showRerank
      ? `<span class="count-text">${text}</span><span class="rerank-badge">⚡ RERANK</span>`
      : `<span class="count-text">${text}</span>`;

    resultCount.innerHTML = html;
  }

  function formatRelativeTime(epochSeconds) {
    const timestampMs = epochSeconds * 1000;
    const now = Date.now();
    const diff = Math.max(0, now - timestampMs);
    const seconds = Math.floor(diff / 1000);

    if (seconds < 45) {
      return 'just now';
    }
    if (seconds < 90) {
      return 'a minute ago';
    }
    const minutes = Math.floor(seconds / 60);
    if (minutes < 60) {
      return `${minutes} min${minutes === 1 ? '' : 's'} ago`;
    }
    const hours = Math.floor(minutes / 60);
    if (hours < 24) {
      return `${hours} hour${hours === 1 ? '' : 's'} ago`;
    }
    const days = Math.floor(hours / 24);
    if (days < 7) {
      return `${days} day${days === 1 ? '' : 's'} ago`;
    }
    const weeks = Math.floor(days / 7);
    if (weeks < 5) {
      return `${weeks} week${weeks === 1 ? '' : 's'} ago`;
    }
    const months = Math.floor(days / 30);
    if (months < 12) {
      return `${months} month${months === 1 ? '' : 's'} ago`;
    }
    const years = Math.floor(days / 365);
    return `${years} year${years === 1 ? '' : 's'} ago`;
  }

  function formatBytes(bytes) {
    if (!bytes || bytes <= 0) {
      return '0 B';
    }
    const units = ['B', 'KB', 'MB', 'GB', 'TB'];
    const exponent = Math.min(units.length - 1, Math.floor(Math.log(bytes) / Math.log(1024)));
    const value = bytes / Math.pow(1024, exponent);
    return `${value.toFixed(value >= 10 || exponent === 0 ? 0 : 1)} ${units[exponent]}`;
  }
})();


--- ck-vscode/src/mcpAdapter.ts ---
import { ChildProcessWithoutNullStreams, spawn } from 'child_process';
import * as path from 'path';
import * as readline from 'readline';
import * as vscode from 'vscode';
import packageJson from '../package.json';
import { CkConfig, IndexStatus, SearchOptions, SearchResponse, SearchResult, IndexProgressUpdate } from './types';

interface PendingRequest {
  resolve: (value: any) => void;
  reject: (error: Error) => void;
}

interface NotificationMessage {
  method?: string;
  params?: any;
}

export class CkMcpAdapter {
  private child: ChildProcessWithoutNullStreams | undefined;
  private readlineInterface: readline.Interface | undefined;
  private pendingRequests = new Map<number, PendingRequest>();
  private nextId = 1;
  private initializationPromise: Promise<void> | undefined;
  private outputChannel = vscode.window.createOutputChannel('ck MCP');
  private progressEmitter = new vscode.EventEmitter<IndexProgressUpdate>();

  public readonly onIndexProgress = this.progressEmitter.event;

  constructor(private readonly config: CkConfig, private readonly indexRoot: string) {}

  async search(options: SearchOptions): Promise<SearchResponse> {
    await this.ensureServer();

    const toolName = this.getToolName(options.mode);
    const resolvedPath = this.resolvePath(options.path);

    let requestBody: Record<string, unknown>;

    if (toolName === 'regex_search') {
      requestBody = {
        pattern: options.query,
        path: resolvedPath,
        ignore_case: options.caseInsensitive ?? false,
        context: options.contextLines,
        include_patterns: options.includePatterns,
        exclude_patterns: options.excludePatterns,
        respect_gitignore: options.respectGitignore ?? true,
        use_default_excludes: options.useDefaultExcludes ?? true,
        whole_word: options.wholeWord ?? false,
        fixed_string: options.fixedString ?? false,
        page_size: options.pageSize,
        include_snippet: options.includeSnippet ?? true,
        snippet_length: options.contextLines,
        cursor: options.cursor
      };
    } else {
      requestBody = {
        query: options.query,
        path: resolvedPath,
        top_k: options.topK,
        threshold: options.threshold,
        page_size: options.pageSize,
        include_patterns: options.includePatterns,
        exclude_patterns: options.excludePatterns,
        respect_gitignore: options.respectGitignore ?? true,
        use_default_excludes: options.useDefaultExcludes ?? true,
        rerank: options.rerank ?? (options.mode === 'semantic' || options.mode === 'hybrid'),
        rerank_model: options.rerankModel,
        case_insensitive: options.caseInsensitive ?? false,
        context_lines: options.contextLines,
        before_context_lines: options.beforeContextLines,
        after_context_lines: options.afterContextLines,
        include_snippet: options.includeSnippet ?? true,
        cursor: options.cursor
      };
    }

    const response = await this.callTool(toolName, requestBody);
    return this.transformSearchResponse(response);
  }

  async getIndexStatus(pathToCheck: string): Promise<IndexStatus> {
    await this.ensureServer();
    const response = await this.callTool('index_status', {
      path: this.resolvePath(pathToCheck)
    });

    const status = response?.index_status ?? response?.indexStatus ?? {};
    return {
      exists: Boolean(status.index_exists ?? status.indexed ?? false),
      path: status.path ?? pathToCheck,
      totalFiles: status.total_files,
      totalChunks: status.total_chunks,
      lastModified: status.last_modified,
      indexPath: status.index_path,
      indexSizeBytes: status.index_size_bytes,
      estimatedFileCount: status.estimated_file_count,
      cacheHit: status.cache_hit
    };
  }

  async reindex(
    pathToReindex: string,
    progress?: vscode.Progress<{ message?: string; increment?: number }>
  ): Promise<void> {
    await this.ensureServer();
    if (progress) {
      progress.report({ message: 'Starting MCP reindex…' });
    }
    await this.callTool('reindex', {
      path: this.resolvePath(pathToReindex),
      force: false
    });
    if (progress) {
      progress.report({ message: 'Reindex complete' });
    }
  }

  async getDefaultCkignoreContent(_indexRoot: string): Promise<string> {
    await this.ensureServer();
    const response = await this.callTool('default_ckignore', {});
    const content = response?.ckignore ?? response?.content ?? response;
    if (typeof content === 'string') {
      return content;
    }
    throw new Error('MCP server did not return default .ckignore content');
  }

  dispose(): void {
    this.shutdownChild('MCP server disposed');
    this.outputChannel.dispose();
    this.progressEmitter.dispose();
  }

  private resolvePath(targetPath: string): string {
    if (!targetPath) {
      return this.indexRoot;
    }
    if (path.isAbsolute(targetPath)) {
      return targetPath;
    }
    return path.join(this.indexRoot, targetPath);
  }

  private getToolName(mode: SearchOptions['mode']): string {
    switch (mode) {
      case 'semantic':
        return 'semantic_search';
      case 'hybrid':
        return 'hybrid_search';
      case 'regex':
        return 'regex_search';
      default:
        return 'lexical_search';
    }
  }

  private async ensureServer(): Promise<void> {
    if (this.initializationPromise) {
      return this.initializationPromise;
    }

    if (!this.child || this.child.killed) {
      this.spawnServer();
    }

    this.initializationPromise = this.initializeServer();
    return this.initializationPromise;
  }

  private spawnServer(): void {
    this.shutdownChild('MCP server restarted');
    const command = this.config.mcpCommand || this.config.cliPath || 'ck';
    const args = this.config.mcpArgs && this.config.mcpArgs.length > 0 ? this.config.mcpArgs : ['--serve'];

    this.child = spawn(command, args, {
      cwd: this.indexRoot,
      stdio: ['pipe', 'pipe', 'pipe']
    });

    this.child.on('error', (err) => {
      vscode.window.showErrorMessage(`ck MCP server failed: ${err.message}`);
      this.rejectAllPending(err);
    });

    this.child.on('exit', (code, signal) => {
      const message = code !== null ? `exit code ${code}` : `signal ${signal}`;
      this.outputChannel.appendLine(`[ck-mcp] server exited (${message})`);
      this.rejectAllPending(new Error(`MCP server exited (${message})`));
      this.child = undefined;
      this.initializationPromise = undefined;
    });

    if (this.child.stderr) {
      this.child.stderr.setEncoding('utf8');
      this.child.stderr.on('data', (data: string) => {
        data
          .split(/\r?\n/)
          .filter(Boolean)
          .forEach((line) => this.outputChannel.appendLine(`[stderr] ${line}`));
      });
    }

    if (this.child.stdout) {
      this.child.stdout.setEncoding('utf8');
      this.readlineInterface = readline.createInterface({ input: this.child.stdout });
      this.readlineInterface.on('line', (line) => this.handleLine(line));
    }

    this.outputChannel.appendLine('[ck-mcp] server spawned');
  }

  private async initializeServer(): Promise<void> {
    try {
      const protocolVersion = '2024-11-05';
      await this.sendRequestInternal('initialize', {
        protocolVersion,
        capabilities: {},
        clientInfo: {
          name: 'ck-vscode',
          version: packageJson.version ?? '0.0.0'
        }
      });
      await this.sendNotification('notifications/initialized', {});
      this.outputChannel.appendLine('[ck-mcp] initialization complete');
    } catch (error) {
      this.outputChannel.appendLine(`[ck-mcp] initialization failed: ${(error as Error).message}`);
      this.shutdownChild('MCP server initialization failed');
      throw error;
    }
  }

  private async callTool(name: string, args: Record<string, unknown>): Promise<any> {
    const rawResult = await this.sendRequest('tools/call', {
      name,
      arguments: args
    });

    const isError = Boolean(rawResult?.is_error ?? rawResult?.isError);
    if (isError) {
      const message = typeof rawResult === 'object' && rawResult?.content?.[0]?.text
        ? rawResult.content[0].text
        : 'Unknown MCP error';
      throw new Error(message);
    }

    return rawResult?.structured_content ?? rawResult?.structuredContent ?? rawResult;
  }

  private transformSearchResponse(raw: any): SearchResponse {
    const matches = Array.isArray(raw?.results?.matches) ? raw.results.matches : [];
    const results: SearchResult[] = matches.map((match: any) => {
      const filePath = match?.file?.path ?? '';
      const span = match?.match?.span ?? {};
      const lineStart = span.line_start ?? 1;
      const lineEnd = span.line_end ?? lineStart;
      return {
        file: filePath,
        lineStart,
        lineEnd,
        byteStart: span.byte_start ?? 0,
        byteEnd: span.byte_end ?? 0,
        preview: match?.match?.content ?? '',
        score: match?.match?.score,
        language: match?.file?.language ?? undefined,
        absolutePath: filePath
      };
    });

    return {
      results,
      count: raw?.results?.count ?? results.length,
      totalCount: raw?.results?.total_count ?? results.length,
      hasMore: Boolean(raw?.results?.has_more),
      nextCursor: raw?.pagination?.next_cursor ?? undefined,
      searchTimeMs: raw?.metadata?.search_time_ms
    };
  }

  private async sendRequest(method: string, params?: Record<string, unknown>): Promise<any> {
    await this.ensureServer();
    return this.sendRequestInternal(method, params ?? {});
  }

  private sendRequestInternal(method: string, params?: Record<string, unknown>): Promise<any> {
    if (!this.child || !this.child.stdin) {
      throw new Error('MCP server is not running');
    }

    const id = this.nextId++;
    const payload = {
      jsonrpc: '2.0',
      id,
      method,
      params
    };

    return new Promise((resolve, reject) => {
      this.pendingRequests.set(id, { resolve, reject });
      this.writeMessage(payload).catch((error) => {
        this.pendingRequests.delete(id);
        reject(error);
      });
    });
  }

  private async sendNotification(method: string, params?: Record<string, unknown>): Promise<void> {
    const payload = {
      jsonrpc: '2.0',
      method,
      params
    };

    try {
      await this.writeMessage(payload);
    } catch (error) {
      this.outputChannel.appendLine(`[ck-mcp] failed to send notification ${method}: ${(error as Error).message}`);
      throw error;
    }
  }

  private handleLine(line: string): void {
    const trimmed = line.trim();
    if (!trimmed) {
      return;
    }

    let message: any;
    try {
      message = JSON.parse(trimmed);
    } catch (error) {
      this.outputChannel.appendLine(`[ck-mcp] failed to parse JSON: ${trimmed}`);
      return;
    }

    if (typeof message.id !== 'undefined') {
      const pending = this.pendingRequests.get(message.id);
      if (!pending) {
        return;
      }
      this.pendingRequests.delete(message.id);

      if (Object.prototype.hasOwnProperty.call(message, 'error')) {
        const errorData = message.error?.data?.details ?? message.error?.message ?? 'Unknown MCP error';
        pending.reject(new Error(errorData));
      } else {
        pending.resolve(message.result);
      }
      return;
    }

    this.handleNotification(message);
  }

  private handleNotification(notification: NotificationMessage): void {
    if (!notification?.method) {
      return;
    }

    switch (notification.method) {
      case '$/progress':
        if (notification.params?.message) {
          this.outputChannel.appendLine(`[ck-mcp] ${notification.params.message}`);
        }
        this.progressEmitter.fire({
          message: notification.params?.message,
          progress: notification.params?.progress,
          total: notification.params?.total,
          source: 'mcp',
          timestamp: Date.now()
        });
        break;
      default:
        this.outputChannel.appendLine(`[ck-mcp] notification ${notification.method}`);
    }
  }

  private rejectAllPending(error: Error): void {
    this.pendingRequests.forEach(({ reject }) => reject(error));
    this.pendingRequests.clear();
  }

  private shutdownChild(message?: string): void {
    if (this.readlineInterface) {
      this.readlineInterface.close();
      this.readlineInterface = undefined;
    }
    if (this.child && !this.child.killed) {
      this.child.kill();
    }
    this.child = undefined;
    this.initializationPromise = undefined;
    if (this.pendingRequests.size > 0) {
      const error = new Error(message ?? 'MCP server stopped');
      this.rejectAllPending(error);
    }
  }

  private writeMessage(payload: Record<string, unknown>): Promise<void> {
    if (!this.child || !this.child.stdin) {
      return Promise.reject(new Error('MCP server is not running'));
    }

    return new Promise((resolve, reject) => {
      this.child!.stdin.write(`${JSON.stringify(payload)}\n`, (err) => {
        if (err) {
          reject(err);
        } else {
          resolve();
        }
      });
    });
  }
}


--- ck-vscode/src/searchPanel.ts ---
/**
 * Webview panel provider for the ck search interface
 */

import * as vscode from 'vscode';
import * as path from 'path';
import { CkCliAdapter } from './cliAdapter';
import { CkMcpAdapter } from './mcpAdapter';
import { SearchMode, SearchOptions, SearchResult, CkConfig, SearchResponse, IndexStatus, IndexProgressUpdate } from './types';

interface SearchBackend {
  search(options: SearchOptions): Promise<SearchResponse>;
  reindex(path: string, progress?: vscode.Progress<{ message?: string; increment?: number }>): Promise<void>;
  getIndexStatus(path: string): Promise<IndexStatus>;
  dispose(): void;
  onIndexProgress?: vscode.Event<IndexProgressUpdate>;
  getDefaultCkignoreContent?(indexRoot: string): Promise<string>;
}

export class CkSearchPanel implements vscode.WebviewViewProvider {
  public static readonly viewType = 'ck.searchView';

  private _view?: vscode.WebviewView;
  private adapter: SearchBackend | undefined;
  private config: CkConfig;
  private currentIndexRoot: string | undefined;
  private adapterDisposables: vscode.Disposable[] = [];

  // Fallback copy of ck-core defaults; extension fetches live content when possible.
  private static readonly FALLBACK_CKIGNORE_CONTENT = `# .ckignore - Default patterns for ck semantic search
# Created automatically during first index
# Syntax: same as .gitignore (glob patterns, ! for negation)

# Images
*.png
*.jpg
*.jpeg
*.gif
*.bmp
*.svg
*.ico
*.webp
*.tiff

# Video
*.mp4
*.avi
*.mov
*.mkv
*.wmv
*.flv
*.webm

# Audio
*.mp3
*.wav
*.flac
*.aac
*.ogg
*.m4a

# Binary/Compiled
*.exe
*.dll
*.so
*.dylib
*.a
*.lib
*.obj
*.o

# Archives
*.zip
*.tar
*.tar.gz
*.tgz
*.rar
*.7z
*.bz2
*.gz

# Data files
*.db
*.sqlite
*.sqlite3
*.parquet
*.arrow

# Config formats (issue #27)
*.json
*.yaml
*.yml

# Add your custom patterns below this line
`;

  constructor(
    private readonly _extensionUri: vscode.Uri,
    config: CkConfig
  ) {
    this.config = config;
  }

  public resolveWebviewView(
    webviewView: vscode.WebviewView,
    context: vscode.WebviewViewResolveContext,
    _token: vscode.CancellationToken
  ) {
    this._view = webviewView;

    webviewView.webview.options = {
      enableScripts: true,
      localResourceRoots: [
        vscode.Uri.joinPath(this._extensionUri, 'webview')
      ]
    };

    webviewView.webview.html = this._getHtmlForWebview(webviewView.webview);

    // Handle messages from the webview
    webviewView.webview.onDidReceiveMessage(async (data) => {
      switch (data.type) {
        case 'search':
          await this.handleSearch(data.query, data.mode, data.includePatterns, data.excludePatterns);
          break;
        case 'openFile':
          await this.openFile(data.file, data.line);
          break;
        case 'reindex':
          await this.handleReindex();
          break;
        case 'getIndexStatus':
          await this.handleIndexStatus();
          break;
        case 'openCkignore':
          await this.openCkignore();
          break;
      }
    });

    // Send initial configuration
    this.sendConfig();
  }

  /**
   * Perform a search
   */
  public async search(query: string, mode?: SearchMode, includePatterns?: string[], excludePatterns?: string[]) {
    if (!this._view) {
      return;
    }

    const searchMode = mode || this.config.defaultMode;

    // Show searching state
    this._view.webview.postMessage({
      type: 'searchStarted',
      query,
      mode: searchMode
    });

    try {
      const workspaceFolder = vscode.workspace.workspaceFolders?.[0];
      if (!workspaceFolder) {
        throw new Error('No workspace folder open');
      }

      const indexRoot = this.resolveIndexRoot(workspaceFolder);
      const options: SearchOptions = {
        query,
        mode: searchMode,
        path: indexRoot,
        topK: this.config.topK,
        threshold: this.config.threshold,
        pageSize: this.config.pageSize,
        contextLines: this.config.contextLines,
        includePatterns,
        excludePatterns,
        rerank: searchMode === 'semantic' || searchMode === 'hybrid',
        includeSnippet: true,
        respectGitignore: true,
        useDefaultExcludes: true,
        beforeContextLines: this.config.contextLines,
        afterContextLines: this.config.contextLines
      };

      const response = await this.getAdapter(indexRoot).search(options);

      // Make file paths relative to workspace
      const workspaceRoot = workspaceFolder.uri.fsPath;
      const relativeResults = response.results.map(r => ({
        ...r,
        file: r.file.startsWith(workspaceRoot)
          ? r.file.substring(workspaceRoot.length + 1)
          : r.file,
        absolutePath: r.file // Keep absolute for opening
      }));

      // Send results to webview
      this._view.webview.postMessage({
        type: 'searchResults',
        results: relativeResults,
        count: response.count,
        totalCount: response.totalCount,
        hasMore: response.hasMore,
        searchTimeMs: response.searchTimeMs ?? response.searchTime,
        nextCursor: response.nextCursor
      });
    } catch (error) {
      this._view.webview.postMessage({
        type: 'searchError',
        error: error instanceof Error ? error.message : String(error)
      });
    }
  }

  /**
   * Reindex the workspace
   */
  public async reindex() {
    const workspaceFolder = vscode.workspace.workspaceFolders?.[0];
    if (!workspaceFolder) {
      vscode.window.showErrorMessage('No workspace folder open');
      return;
    }

    const indexRoot = this.resolveIndexRoot(workspaceFolder);
    const adapter = this.getAdapter(indexRoot);
    const source: IndexProgressUpdate['source'] = this.config.mode === 'mcp' ? 'mcp' : 'cli';

    if (this._view) {
      this._view.webview.postMessage({
        type: 'indexProgress',
        update: {
          message: 'Starting reindex…',
          source,
          timestamp: Date.now()
        }
      });
    }

    await vscode.window.withProgress(
      {
        location: vscode.ProgressLocation.Notification,
        title: 'Reindexing with ck',
        cancellable: false
      },
      async (progress) => {
        try {
          await adapter.reindex(indexRoot, progress);
          vscode.window.showInformationMessage('ck: Reindexing complete');

          // Update index status in UI
          await this.handleIndexStatus();
        } catch (error) {
          vscode.window.showErrorMessage(
            `ck: Reindexing failed: ${error instanceof Error ? error.message : String(error)}`
          );
        }
      }
    );
  }

  /**
   * Update configuration
   */
  public updateConfig(config: CkConfig) {
    this.config = config;
    this.resetAdapter();
    this.sendConfig();
  }

  private async handleSearch(query: string, mode: SearchMode, includePatterns?: string[], excludePatterns?: string[]) {
    await this.search(query, mode, includePatterns, excludePatterns);
  }

  private async handleReindex() {
    await this.reindex();
  }

  private async handleIndexStatus() {
    const workspaceFolder = vscode.workspace.workspaceFolders?.[0];
    if (!workspaceFolder || !this._view) {
      return;
    }

    try {
      const indexRoot = this.resolveIndexRoot(workspaceFolder);
      const status = await this.getAdapter(indexRoot).getIndexStatus(indexRoot);

      this._view.webview.postMessage({
        type: 'indexStatus',
        status
      });
    } catch (error) {
      console.error('Failed to get index status:', error);
    }
  }

  private async openFile(file: string, line: number) {
    try {
      // If file is relative, make it absolute
      const workspaceFolder = vscode.workspace.workspaceFolders?.[0];
      const absolutePath = path.isAbsolute(file)
        ? file
        : workspaceFolder
          ? path.join(workspaceFolder.uri.fsPath, file)
          : file;

      const uri = vscode.Uri.file(absolutePath);
      const document = await vscode.workspace.openTextDocument(uri);
      const editor = await vscode.window.showTextDocument(document);

      // Jump to line
      const position = new vscode.Position(Math.max(0, line - 1), 0);
      editor.selection = new vscode.Selection(position, position);
      editor.revealRange(
        new vscode.Range(position, position),
        vscode.TextEditorRevealType.InCenter
      );

      // Highlight the line briefly
      const decoration = vscode.window.createTextEditorDecorationType({
        backgroundColor: new vscode.ThemeColor('editor.findMatchHighlightBackground'),
        isWholeLine: true
      });

      editor.setDecorations(decoration, [new vscode.Range(position, position)]);

      setTimeout(() => {
        decoration.dispose();
      }, 2000);
    } catch (error) {
      vscode.window.showErrorMessage(
        `Failed to open file: ${error instanceof Error ? error.message : String(error)}`
      );
    }
  }

  private sendConfig() {
    if (!this._view) {
      return;
    }

    this._view.webview.postMessage({
      type: 'config',
      config: this.config
    });
  }

  private _getHtmlForWebview(webview: vscode.Webview): string {
    // Get URIs for webview resources
    const scriptUri = webview.asWebviewUri(
      vscode.Uri.joinPath(this._extensionUri, 'webview', 'main.js')
    );
    const styleUri = webview.asWebviewUri(
      vscode.Uri.joinPath(this._extensionUri, 'webview', 'styles.css')
    );

    // Use a nonce for CSP
    const nonce = getNonce();

    return `<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="Content-Security-Policy" content="default-src 'none'; style-src ${webview.cspSource} 'unsafe-inline'; script-src 'nonce-${nonce}';">
  <link href="${styleUri}" rel="stylesheet">
  <title>ck Search</title>
</head>
<body>
  <div class="search-container">
    <div class="search-header">
      <div class="search-input-wrapper">
        <input type="text" id="searchInput" placeholder="Search code..." autofocus>
      </div>
      <div class="search-controls">
        <select id="modeSelector">
          <option value="hybrid">Hybrid</option>
          <option value="semantic">Semantic</option>
          <option value="regex">Regex</option>
        </select>
        <button id="reindexButton" title="Reindex workspace">
          <span class="codicon codicon-refresh"></span>
        </button>
      </div>
      <div class="filter-controls">
        <input type="text" id="includeInput" placeholder="files to include: *.ts *.js (comma or space separated)" class="filter-input">
        <input type="text" id="excludeInput" placeholder="files to exclude: *.html *.md (comma or space separated)" class="filter-input">
      </div>
      <div class="index-meta">
        <div class="index-status" id="indexStatus" role="status">
          <span class="status-indicator"></span>
          <span class="status-text">Checking index...</span>
        </div>
        <div class="index-actions">
          <button id="refreshStatusButton" class="icon-button" title="Refresh index status">
            <span class="codicon codicon-refresh"></span>
          </button>
          <button id="ckignoreButton" class="link-button" title="Open .ckignore rules">.ckignore</button>
        </div>
      </div>
      <div class="index-progress hidden" id="indexProgress">
        <span class="codicon codicon-sync"></span>
        <span class="progress-text">Indexing...</span>
      </div>
    </div>

    <div class="search-body">
      <div id="loadingIndicator" class="loading hidden">
        <div class="spinner"></div>
        <span>Searching...</span>
      </div>

      <div id="resultsContainer" class="results-container">
        <div class="empty-state">
          <p>Enter a search query to find code</p>
          <p class="hint">Try searching for concepts like "error handling" or "database connection"</p>
        </div>
      </div>

      <div id="errorContainer" class="error-container hidden"></div>
    </div>

    <div class="search-footer">
      <div id="resultCount" class="result-count"></div>
    </div>
  </div>

  <script nonce="${nonce}" src="${scriptUri}"></script>
</body>
</html>`;
  }

  public dispose() {
    if (this.adapter) {
      this.adapter.dispose();
      this.adapter = undefined;
    }
    this.disposeAdapterEvents();
  }

  private resetAdapter() {
    if (this.adapter) {
      this.adapter.dispose();
    }
    this.adapter = undefined;
    this.currentIndexRoot = undefined;
    this.disposeAdapterEvents();
  }

  private getAdapter(indexRoot: string): SearchBackend {
    if (!this.adapter || this.currentIndexRoot !== indexRoot) {
      if (this.adapter) {
        this.adapter.dispose();
      }
      this.disposeAdapterEvents();
      this.adapter = this.createAdapter(this.config, indexRoot);
      this.currentIndexRoot = indexRoot;
      this.registerAdapterEvents(this.adapter);
    }
    return this.adapter;
  }

  private createAdapter(config: CkConfig, indexRoot: string): SearchBackend {
    if (config.mode === 'mcp') {
      return new CkMcpAdapter(config, indexRoot);
    }
    return new CkCliAdapter(config.cliPath);
  }

  private registerAdapterEvents(adapter: SearchBackend) {
    if (adapter.onIndexProgress) {
      const disposable = adapter.onIndexProgress((update) => {
        if (this._view) {
          this._view.webview.postMessage({
            type: 'indexProgress',
            update
          });
        }
      });
      this.adapterDisposables.push(disposable);
    }
  }

  private disposeAdapterEvents() {
    this.adapterDisposables.forEach((disposable) => disposable.dispose());
    this.adapterDisposables = [];
  }

  private resolveIndexRoot(workspaceFolder?: vscode.WorkspaceFolder): string {
    const workspacePath = workspaceFolder?.uri.fsPath ?? process.cwd();
    const template = this.config.indexRoot ?? '${workspaceFolder}';
    const substituted = template.replace('${workspaceFolder}', workspacePath);
    return path.isAbsolute(substituted)
      ? substituted
      : path.resolve(workspacePath, substituted);
  }

  private async openCkignore() {
    const workspaceFolder = vscode.workspace.workspaceFolders?.[0];
    if (!workspaceFolder) {
      vscode.window.showErrorMessage('No workspace folder open');
      return;
    }

    const ckignoreUri = vscode.Uri.joinPath(workspaceFolder.uri, '.ckignore');
    let exists = true;
    try {
      await vscode.workspace.fs.stat(ckignoreUri);
    } catch (error) {
      exists = false;
    }

    if (!exists) {
      const selection = await vscode.window.showInformationMessage(
        'No .ckignore file found for this workspace. Create one with the default ck patterns?',
        'Create .ckignore',
        'Cancel'
      );

      if (selection !== 'Create .ckignore') {
        return;
      }

      const indexRoot = this.resolveIndexRoot(workspaceFolder);
      const adapter = this.getAdapter(indexRoot);
      let defaultContent = CkSearchPanel.FALLBACK_CKIGNORE_CONTENT;

      if (adapter.getDefaultCkignoreContent) {
        try {
          const fetched = await adapter.getDefaultCkignoreContent(indexRoot);
          if (typeof fetched === 'string' && fetched.trim().length > 0) {
            defaultContent = fetched;
          }
        } catch (error) {
          console.warn('Failed to fetch default .ckignore content from ck backend:', error);
        }
      }

      if (!defaultContent.endsWith('\n')) {
        defaultContent = `${defaultContent}\n`;
      }

      await vscode.workspace.fs.writeFile(
        ckignoreUri,
        Buffer.from(defaultContent, 'utf8')
      );
      vscode.window.showInformationMessage('.ckignore created with default ck patterns');
    }

    const document = await vscode.workspace.openTextDocument(ckignoreUri);
    await vscode.window.showTextDocument(document, { preview: false });
  }
}

function getNonce(): string {
  let text = '';
  const possible = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
  for (let i = 0; i < 32; i++) {
    text += possible.charAt(Math.floor(Math.random() * possible.length));
  }
  return text;
}


--- ck-vscode/src/types.ts ---
/**
 * Type definitions for ck search integration
 */

export type SearchMode = 'hybrid' | 'semantic' | 'regex';

export interface SearchResult {
  file: string;
  lineStart: number;
  lineEnd: number;
  byteStart: number;
  byteEnd: number;
  preview: string;
  score?: number;
  language?: string;
  absolutePath?: string; // Absolute path for opening (when file is relative)
}

export interface SearchOptions {
  query: string;
  mode: SearchMode;
  path: string;
  topK?: number;
  threshold?: number;
  pageSize?: number;
  caseInsensitive?: boolean;
  contextLines?: number;
  includePatterns?: string[];
  excludePatterns?: string[];
  rerank?: boolean;
  rerankModel?: string;
  respectGitignore?: boolean;
  useDefaultExcludes?: boolean;
  includeSnippet?: boolean;
  beforeContextLines?: number;
  afterContextLines?: number;
  cursor?: string;
  wholeWord?: boolean;
  fixedString?: boolean;
}

export interface IndexStatus {
  exists: boolean;
  path: string;
  totalFiles?: number;
  totalChunks?: number;
  lastModified?: number;
  indexPath?: string;
  indexSizeBytes?: number;
  estimatedFileCount?: number;
  cacheHit?: boolean;
}

export interface SearchResponse {
  results: SearchResult[];
  count: number;
  totalCount: number;
  hasMore: boolean;
  searchTime?: number;
  nextCursor?: string;
  searchTimeMs?: number;
}

export interface CkConfig {
  mode: 'cli' | 'mcp';
  cliPath: string;
  mcpCommand: string;
  mcpArgs: string[];
  indexRoot: string;
  defaultMode: SearchMode;
  pageSize: number;
  threshold: number;
  topK: number;
  contextLines: number;
}

export type IndexProgressSource = 'cli' | 'mcp';

export interface IndexProgressUpdate {
  message?: string;
  progress?: number;
  total?: number;
  source: IndexProgressSource;
  timestamp?: number;
}
