Metadata-Version: 2.3
Name: testiq
Version: 0.2.0
Summary: TestIQ - Intelligent test analysis for finding duplicate and redundant tests using coverage analysis
Keywords: testing,test-automation,coverage,duplicate-detection,test-quality,ci-cd
Author: Kiran K Kotari
Author-email: Kiran K Kotari <kirankotari@live.com>
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: click>=8.1.7
Requires-Dist: coverage>=7.4.0
Requires-Dist: rich>=13.7.0
Requires-Dist: pyyaml>=6.0.1
Requires-Dist: tomli>=2.0.1 ; python_full_version < '3.11'
Requires-Dist: pytest>=7.4.0
Requires-Dist: pytest>=7.4.0 ; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0 ; extra == 'dev'
Requires-Dist: ruff>=0.1.0 ; extra == 'dev'
Requires-Dist: black>=23.0.0 ; extra == 'dev'
Requires-Dist: mypy>=1.7.0 ; extra == 'dev'
Requires-Dist: types-click>=7.1.0 ; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0.0 ; extra == 'dev'
Requires-Python: >=3.9
Project-URL: Homepage, https://github.com/testiq-dev/testiq
Project-URL: Documentation, https://github.com/testiq-dev/testiq/tree/main/docs
Project-URL: Repository, https://github.com/testiq-dev/testiq
Project-URL: Issues, https://github.com/testiq-dev/testiq/issues
Provides-Extra: dev
Description-Content-Type: text/markdown

# TestIQ - Intelligent Test Analysis

[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/downloads/)
[![Version](https://img.shields.io/badge/version-0.2.0-blue.svg)](https://github.com/testiq-dev/testiq)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![Test Coverage](https://img.shields.io/badge/coverage-81%25-brightgreen.svg)](htmlcov/index.html)
[![Tests](https://img.shields.io/badge/tests-190%2F190%20passing-brightgreen.svg)](tests/)
[![Quality](https://img.shields.io/badge/quality-A%2B%20(99.9%2F100)-brightgreen.svg)](reports/duplicates.html)

**TestIQ v0.2.0** - An enterprise-grade Python library that analyzes test coverage to identify duplicate, redundant, and inefficient tests. Save time, reduce CI costs, and improve test suite quality with intelligent analysis.

## ✨ What's New in v0.2.0

- **🎯 Enhanced Code Quality** - Reduced cognitive complexity, better maintainability
- **📝 Constants Management** - String literals extracted to constants (best practices)
- **🔧 Improved Type Support** - Type stubs for better IDE autocomplete (click, PyYAML)
- **✅ Better Testing** - Float comparison fixes, proper unused variable handling
- **🧹 Clean Repository** - No sample/dummy data, all reports generated by testiq
- **📦 pytest Included** - Now a core dependency (no separate install needed)

---

## 🚀 Key Features

- **🎯 Duplicate Detection** - Find exact duplicates, subsets, and similar tests
- **📊 Quality Scoring** - A-F grading with actionable recommendations (improved in v0.2.0)
- **🚦 CI/CD Integration** - Quality gates, baselines, and trend tracking
- **⚡ Performance** - Parallel processing and caching for large test suites
- **🔌 Extensible** - Plugin system with custom hooks
- **📝 Multiple Formats** - HTML, CSV, and Markdown reports (all formats in sync)
- **🧪 pytest Integration** - Built-in plugin for per-test coverage tracking (now included!)

---

## 📦 Installation

```bash
pip install testiq
```

---

## 🚀 How to Use TestIQ

### Quick Start

**See it in action immediately:**

```bash
# Run demo with sample data
testiq demo
```

**Generate per-test coverage data:**

TestIQ needs **per-test** coverage (which lines each test executes). Use our pytest plugin:

```bash
# Run tests with TestIQ plugin (easiest method - recommended)
pytest --testiq-output=testiq_coverage.json

# Or specify your test directory
pytest tests/ --testiq-output=testiq_coverage.json

# Then analyze with TestIQ
testiq analyze testiq_coverage.json --format html --output report.html
```

**Alternative methods:**

```bash
# Method 2: Use pytest coverage contexts (requires pytest-cov)
pytest --cov=src --cov-context=test --cov-report=json
python -m testiq.coverage_converter coverage.json --with-contexts -o testiq_coverage.json

# Method 3: Convert standard pytest coverage (limited - aggregated only)
pytest --cov=src --cov-report=json
python -m testiq.coverage_converter coverage.json -o testiq_coverage.json
```

**Manual sample data:**

```bash
# Create sample coverage data (for testing)
cat > testiq_coverage.json << 'EOF'
{
  "test_login_success": {
    "auth.py": [10, 11, 12, 15, 20, 25],
    "user.py": [5, 6, 7, 8]
  },
  "test_login_failure": {
    "auth.py": [10, 11, 12, 15, 16],
    "user.py": [5, 6]
  }
}
EOF

# Analyze the sample data
testiq analyze testiq_coverage.json
```

> **📖 See [Pytest Integration Guide](docs/pytest-integration.md) for complete setup instructions**

### CLI Usage

#### Global Options

```bash
# Check version
testiq --version

# Use custom config file
testiq --config my-config.yaml analyze testiq_coverage.json

# Set log level
testiq --log-level DEBUG analyze testiq_coverage.json

# Save logs to file
testiq --log-file testiq.log analyze testiq_coverage.json

# Run demo with sample data
testiq demo
```

#### 1. Analyze Your Tests

```bash
# Basic analysis with terminal output (text format)
testiq analyze testiq_coverage.json

# With custom similarity threshold (default: 0.3)
testiq analyze testiq_coverage.json --threshold 0.8

# Generate beautiful HTML report
testiq analyze testiq_coverage.json --format html --output reports/report.html

# CSV export for spreadsheet analysis
testiq analyze testiq_coverage.json --format csv --output reports/results.csv

# JSON output for automation
testiq analyze testiq_coverage.json --format json --output reports/results.json

# Markdown format
testiq analyze testiq_coverage.json --format markdown --output reports/results.md

# Get quality score and recommendations
testiq quality-score testiq_coverage.json --output reports/quality.txt
```

#### 2. CI/CD Integration

```bash
# Quality gates (exit code 2 if failed, 1 if duplicates found, 0 if success)
testiq analyze testiq_coverage.json --quality-gate --max-duplicates 5

# Save baseline for future comparisons
testiq analyze testiq_coverage.json --save-baseline my-baseline

# Compare against baseline (fail if quality worsens)
testiq analyze testiq_coverage.json --quality-gate --baseline my-baseline

# Combined: quality gate with threshold and baseline
testiq analyze testiq_coverage.json \
  --quality-gate \
  --max-duplicates 5 \
  --threshold 0.8 \
  --baseline production \
  --format html \
  --output reports/ci-report.html

# Manage baselines
testiq baseline list                    # List all saved baselines
testiq baseline show my-baseline        # Show baseline details
testiq baseline delete old-baseline     # Delete a baseline
```

#### 3. Python API

```python
from testiq.analyzer import CoverageDuplicateFinder
import json

# Create analyzer with performance options
finder = CoverageDuplicateFinder(
    enable_parallel=True,
    max_workers=4,
    enable_caching=True
)

# Load coverage data
with open('coverage.json') as f:
    coverage_data = json.load(f)

# Add test coverage
for test_name, test_coverage in coverage_data.items():
    finder.add_test_coverage(test_name, test_coverage)

# Find issues
exact_duplicates = finder.find_exact_duplicates()
subset_duplicates = finder.find_subset_duplicates()
similar_tests = finder.find_similar_coverage(threshold=0.8)

# Generate reports
from testiq.reporting import HTMLReportGenerator
html_gen = HTMLReportGenerator(finder)
html_gen.generate(Path("reports/report.html"), threshold=0.8)

# Quality analysis
from testiq.analysis import QualityAnalyzer
analyzer = QualityAnalyzer(finder)
score = analyzer.calculate_score(threshold=0.8)
print(f"Quality Score: {score.overall_score}/100 (Grade: {score.grade})")
```

#### 4. Examples & Testing

**See complete working examples:**
- 📁 **[Python API Examples](examples/python/)** - Complete demonstration of all features
- 📁 **[Bash Examples](examples/bash/)** - Quick CLI testing scripts
- 📁 **[CI/CD Examples](examples/cicd/)** - Jenkins & GitHub Actions integration
- 📖 **[Manual Testing Guide](docs/manual-testing.md)** - Comprehensive testing guide

**CI/CD Integration:**
- [Jenkinsfile Example](examples/cicd/Jenkinsfile) - Complete Jenkins pipeline with quality gates
- [GitHub Actions Example](examples/cicd/github-actions.yml) - Full workflow with error handling

---

### Next Steps

1. **Try the demo:** `testiq demo`
2. **Analyze your tests:** `testiq analyze coverage.json`
3. **Run examples:** `python examples/python/manual_test.py`
4. **Integrate with CI/CD:** See [examples/cicd/](examples/cicd/)

---

## � Understanding TestIQ Results

### Quality Score Components

TestIQ calculates a comprehensive quality score (0-100) with letter grade (A+ to F):

- **Duplication Score (40%)** - Based on exact duplicates found
- **Coverage Efficiency (30%)** - Tests with broad vs. narrow coverage
- **Uniqueness Score (30%)** - Based on similar/subset tests

**Important Notes:**
- **Coverage Efficiency requires source metadata** - Use `pytest --cov` with `--cov-report=json` to generate `.coverage` file with source line info
- **Without source metadata, efficiency score is 0** - This lowers overall grade but doesn't indicate actual problems
- Run `make test-complete` or sequential coverage+TestIQ analysis for full scoring

### Interpreting Duplicate Detection

TestIQ uses **coverage-based duplicate detection** - it identifies tests that execute the same code paths. This can include:

**True Duplicates (Should Review):**
- ✅ Multiple tests with identical coverage and same purpose
- ✅ Copy-pasted tests with minor naming differences
- ✅ Tests that add no unique code coverage value

**False Positives (Expected Behavior):**
- ⚠️ Tests with same *coverage* but different *assertions/logic*
- ⚠️ Tests that exercise different input values (same code path)
- ⚠️ Tests focused on behavior verification vs. code coverage

**Example:** Two tests that both create a dataclass instance will show identical coverage (import paths), but one might test validation while another tests defaults - both are valuable!

**Recommendations:**
1. **Review high-priority duplicates first** - These have highest impact
2. **Check test intent, not just coverage** - Different assertions mean different value
3. **Focus on exact duplicates** - Subset/similar tests may be intentional
4. **Use quality score as a guide** - Not an absolute metric
5. **Combine with test execution time** - Slow duplicate tests are higher priority

### Running Complete Analysis

For best results, run coverage and duplicate detection separately:

```bash
# Option 1: Use make target (recommended)
make test-complete

# Option 2: Run manually
pytest --cov=testiq --cov-report=term --cov-report=html  # Coverage first
pytest --testiq-output=testiq_coverage.json -q            # TestIQ second
testiq analyze testiq_coverage.json --format html --output reports/duplicates.html

# Option 3: Use provided script
./run_complete_analysis.sh
```

**Why separate runs?** Python's `sys.settrace()` allows only ONE active tracer. Running both together causes conflicts (19% coverage vs 81% separate).

---

## 💡 Why TestIQ?

- ✅ **Reduce Test Count** - Remove 10-30% of redundant tests
- ✅ **Faster CI/CD** - Shorter test execution times  
- ✅ **Lower Costs** - Reduced CI resource usage
- ✅ **Better Quality** - Focus on valuable, unique tests
- ✅ **Smart Analysis** - Coverage-based duplicate detection with quality scoring

---

## 📚 Documentation
### Essential Documentation
| Document | Description |
|----------|-------------|
| **[Interpreting Results](docs/interpreting-results.md)** | **Guide to understanding TestIQ scores and recommendations** |
| **[CHANGELOG](docs/CHANGELOG.md)** | Version history and v0.2.0 release notes |
| **[Pytest Integration](docs/pytest-integration.md)** | **Generate per-test coverage data** |
| **[CLI Reference](docs/cli-reference.md)** | Command-line interface documentation |
| **[API Reference](docs/api.md)** | Python API documentation |
| **[Contributing](docs/CONTRIBUTING.md)** | How to contribute |
| **[Security Policy](docs/SECURITY.md)** | Security and vulnerability reporting |

### Additional Resources (examples/)
- **Python Examples** - Complete API demonstrations
- **CI/CD Examples** - GitHub Actions, GitLab CI, Jenkins
- **Sample Data** - `examples/sample_coverage.json`ribute |
| **[Changelog](docs/CHANGELOG.md)** | Version history |

---

## 🤝 Contributing

We welcome contributions! See **[CONTRIBUTING.md](docs/CONTRIBUTING.md)** for guidelines.

---

## 📝 License

MIT License - see [LICENSE](LICENSE) for details.

---

## 🔒 Security

For security vulnerabilities, see **[SECURITY.md](docs/SECURITY.md)** for responsible disclosure.

---

**Made with ❤️ by Kiran K Kotari**
