Metadata-Version: 2.4
Name: ngs-ai-agent
Version: 1.0.0
Summary: AI-powered automated NGS analysis pipeline
Home-page: https://github.com/your-org/ngs-ai-agent
Author: NGS AI Agent Team
Author-email: contact@ngs-ai-agent.com
Project-URL: Bug Reports, https://github.com/your-org/ngs-ai-agent/issues
Project-URL: Source, https://github.com/your-org/ngs-ai-agent
Project-URL: Documentation, https://ngs-ai-agent.readthedocs.io/
Keywords: ngs,bioinformatics,ai,genomics,sequencing,pipeline
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.8; extra == "dev"
Requires-Dist: mypy>=0.800; extra == "dev"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license-file
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-python
Dynamic: summary

# NGS AI Agent

An AI-powered agent for fully automated next-generation sequencing (NGS) data analysis, with Deep Mutational Scanning as a proof of concept.

## Features

### 🤖 **AI-Powered Intelligence**
- **Smart Metadata Analysis**: Ollama analyzes experimental descriptions to detect conditions
- **Intelligent File Matching**: AI matches messy sequencer filenames to metadata entries
- **Natural Language Understanding**: Extracts experimental information from human-readable descriptions
- **Multi-layer Matching**: Exact → AI → Fuzzy → Basic detection with robust fallbacks

### 🧬 **Complete NGS Pipeline** 
- **End-to-End Workflow**: FASTQ files → Final reports with AI orchestration
- **Modular Design**: Reusable components for QC, trimming, mapping, and variant calling
- **Deep Mutational Scanning**: Specialized fitness calculation and amino acid effect analysis
- **Quality Control**: Automated QC with FastQC, trimming with Cutadapt, mapping with Bowtie2

### 📊 **Smart Analysis & Visualization**
- **AI-Generated Reports**: Scientific insights and recommendations powered by Ollama
- **Interactive Heatmaps**: Publication-ready visualizations of mutational effects
- **Tabular Metadata**: User-friendly CSV/Excel format for experimental design
- **Privacy-First**: All AI processing happens locally (no cloud dependencies)  

## Quick Start

### 1. Setup Environment

```bash
# Clone and navigate to project
cd ngs-ai-agent

# Check if your existing ai-ngs environment has all required dependencies
python check_environment.py

# If dependencies are missing, install them in your ai-ngs environment:
conda activate ai-ngs
conda install -c conda-forge -c bioconda snakemake fastqc cutadapt bowtie2 samtools bcftools
pip install ollama langchain langchain-community

# Start Ollama (in separate terminal)
ollama serve
ollama pull qwen3-coder:latest
```

### 2. Run Analysis

```bash
# AI-powered analysis with metadata (recommended)
python src/ngs_ai_agent.py run \
  --input-dir /path/to/fastq/files \
  --reference /path/to/reference.fasta \
  --metadata experiment_metadata.csv

# Basic analysis with auto-discovery
python src/ngs_ai_agent.py run --input-dir /path/to/fastq/files --reference /path/to/reference.fasta

# Dry run to check AI configuration
python src/ngs_ai_agent.py run --input-dir /path/to/fastq/files --metadata experiment.csv --dry-run

# Custom cores and config
python src/ngs_ai_agent.py run --input-dir /path/to/fastq/files --cores 16 --metadata experiment.csv
```

### 3. View Results

Results will be generated in the `results/` directory:
- **Final Report**: `results/reports/final_report.html`
- **Fitness Scores**: `results/dms/fitness_scores.csv`
- **Heatmaps**: `results/visualization/dms_heatmap.png` and `.html`

## Project Structure

```
ngs-ai-agent/
├── config/
│   └── config.yaml              # Main configuration
├── workflow/
│   ├── Snakefile               # Main pipeline
│   ├── rules/                  # Modular Snakemake rules
│   │   ├── qc.smk             # Quality control
│   │   ├── trimming.smk       # Read trimming
│   │   ├── mapping.smk        # Read mapping
│   │   ├── variant_calling.smk # Variant calling
│   │   ├── dms_analysis.smk   # DMS fitness calculation
│   │   └── visualization.smk   # Plotting and visualization
│   └── scripts/               # Custom analysis scripts
│       ├── variant_caller.py  # Custom variant caller
│       ├── dms_fitness.py     # Fitness calculation
│       ├── variant_annotation.py
│       ├── create_heatmap.py  # Heatmap generation
│       └── fitness_plots.py   # Additional plots
├── src/
│   ├── ngs_ai_agent.py        # Main CLI interface
│   ├── ai_orchestrator/       # AI components
│   │   └── ollama_client.py   # Ollama integration
│   ├── modules/               # Analysis modules
│   └── visualization/         # Visualization tools
├── data/
│   ├── raw/                   # Input FASTQ files
│   ├── processed/             # Processed data
│   └── test/                  # Test datasets
├── results/                   # Analysis outputs
├── resources/                 # Reference genomes, annotations
├── logs/                      # Pipeline logs
└── environment.yml           # Conda environment
```

## AI Features

### Metadata Detection
The AI automatically analyzes filenames to extract:
- Sample names and IDs
- Time points (T0, T1, day0, day7, etc.)
- Replicate numbers
- Read types (R1/R2 for paired-end)
- Experimental conditions

### Filename Cleaning
Automatically standardizes filenames to:
```
sample1_T0_rep1_R1.fastq.gz
sample1_T0_rep1_R2.fastq.gz
sample2_T1_rep1_R1.fastq.gz
...
```

### Pipeline Configuration
AI determines:
- Single vs paired-end sequencing
- Sample groupings and time series
- Optimal parameters for each analysis step

### Report Generation
AI generates insights including:
- Key findings summary
- Hotspot identification
- Biological implications
- Follow-up recommendations

## Deep Mutational Scanning Pipeline

### 1. Quality Control
- FastQC analysis of raw reads
- MultiQC summary reports

### 2. Read Processing
- Adapter trimming with Cutadapt
- Quality filtering

### 3. Mapping
- Bowtie2 alignment to reference
- Local alignment optimized for DMS

### 4. Variant Calling
- Custom high-sensitivity variant caller
- Optimized for high-frequency variants in DMS

### 5. Fitness Calculation
- Enrichment ratio calculation
- Amino acid effect annotation
- Statistical analysis

### 6. Visualization
- Interactive heatmaps (position vs amino acid)
- Fitness distribution plots
- Coverage analysis

## Configuration

Edit `config/config.yaml` to customize:

```yaml
# Pipeline settings
pipeline:
  threads: 8
  memory_gb: 16

# AI settings
ai:
  model: "llama3.1:8b"
  temperature: 0.1

# Analysis parameters
variant_calling:
  min_coverage: 10
  min_frequency: 0.01

dms:
  fitness_calculation:
    method: "enrichment_ratio"
    pseudocount: 1
```

## Extending the Pipeline

### Adding New Analysis Types

1. Create new rules in `workflow/rules/`
2. Add corresponding scripts in `workflow/scripts/`
3. Update the main `Snakefile`
4. Modify configuration as needed

Example for RNA-seq analysis:
```python
# workflow/rules/rnaseq.smk
rule star_align:
    input:
        reads="results/trimmed/{sample}.fastq.gz"
    output:
        bam="results/mapped/{sample}.bam"
    shell:
        "STAR --runThreadN {threads} --genomeDir {params.index} ..."
```

### Custom AI Prompts

Modify `src/ai_orchestrator/ollama_client.py` to add new AI capabilities:

```python
def detect_experimental_design(self, metadata):
    prompt = f"""
    Analyze this experimental metadata and determine the study design:
    {metadata}
    
    Identify:
    1. Control vs treatment groups
    2. Time course experiments
    3. Dose-response studies
    """
    # ... implementation
```

## Troubleshooting

### Common Issues

1. **Ollama not running**: Start with `ollama serve`
2. **Model not found**: Pull model with `ollama pull qwen3-coder:latest`
3. **Environment issues**: Recreate with `conda env remove -n ai-ngs && python src/ngs_ai_agent.py setup`
4. **Memory issues**: Reduce threads in config or increase system memory

### Logs

Check logs in:
- `logs/`: Snakemake execution logs
- Console output: Real-time pipeline progress

## Dependencies

### Core Tools
- Snakemake (workflow management)
- FastQC (quality control)
- Cutadapt (trimming)
- Bowtie2 (mapping)
- Samtools/BCFtools (file processing)

### AI Components
- Ollama (local LLM server)
- LangChain (AI orchestration)

### Python Libraries
- Pandas, NumPy (data processing)
- Matplotlib, Seaborn, Plotly (visualization)
- Biopython (sequence analysis)
- PyYAML (configuration)

## Citation

If you use NGS AI Agent in your research, please cite:

```
NGS AI Agent: AI-powered automated next-generation sequencing analysis
[Your publication details here]
```

## License

MIT License - see LICENSE file for details.

## Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

## Support

- 📧 Email: [your-email@example.com]
- 🐛 Issues: GitHub Issues page
- 📚 Documentation: [Link to detailed docs]
