Metadata-Version: 2.4
Name: genaiforgenews
Version: 0.1.0
Summary: AI-driven news aggregation: multi-source fetch, AI scoring, bilingual summaries (EN/HI), MCP server.
Project-URL: Homepage, https://github.com/vikas972/genAINews
Project-URL: Documentation, https://vikas972.github.io/genAINews/
Project-URL: Repository, https://github.com/vikas972/genAINews.git
Project-URL: Issues, https://github.com/vikas972/genAINews/issues
Author: Vikas Maurya
License-Expression: MIT
License-File: LICENSE
Keywords: aggregation,ai,hackernews,llm,mcp,news,rss,summarization
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content :: News/Diary
Requires-Python: >=3.11
Requires-Dist: anthropic>=0.39.0
Requires-Dist: beautifulsoup4>=4.12.0
Requires-Dist: ddgs>=7.0.0
Requires-Dist: feedparser>=6.0.11
Requires-Dist: google-genai>=0.3.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: markdown>=3.10.2
Requires-Dist: mcp>=1.0.0
Requires-Dist: openai>=1.54.0
Requires-Dist: pydantic>=2.9.0
Requires-Dist: python-dotenv>=1.0.0
Requires-Dist: rich>=13.9.0
Requires-Dist: tenacity>=9.0.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.24.0; extra == 'dev'
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Description-Content-Type: text/markdown

<div align="center">

# genaiforgenews

**AI curates the tech news. You just read.**

[![Python](https://img.shields.io/badge/python-3.11+-blue.svg?style=flat-square&logo=python&logoColor=white)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg?style=flat-square)](LICENSE)
[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json&style=flat-square)](https://github.com/astral-sh/uv)
[![Daily Summary](https://github.com/vikas972/genAINews/actions/workflows/deploy-docs.yml/badge.svg?style=flat-square)](https://vikas972.github.io/genAINews/)
[![GitHub commit activity](https://img.shields.io/github/commit-activity/m/vikas972/genAINews?style=flat-square)](https://github.com/vikas972/genAINews/commits/master)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)
![Sources Welcome](https://img.shields.io/badge/📡_sources-welcome-f97316?style=flat-square)
<br>

![Claude](https://img.shields.io/badge/Claude-f0daba?style=flat-square&logo=anthropic&logoColor=black)
![GPT](https://img.shields.io/badge/GPT-412991?style=flat-square)
![Gemini](https://img.shields.io/badge/Gemini-8E75B2?style=flat-square&logo=google&logoColor=white)
![DeepSeek](https://img.shields.io/badge/DeepSeek-0A6DC2?style=flat-square)
![Doubao](https://img.shields.io/badge/Doubao-00D6C2?style=flat-square)
![MiniMax](https://img.shields.io/badge/MiniMax-FF6F00?style=flat-square)
![OpenClaw](https://img.shields.io/badge/OpenClaw-C83232?style=flat-square)

genaiforgenews collects news from multiple customizable sources, uses AI to score and filter them, and generates a daily briefing — complete with summaries, community discussions, and background explanations in English and/or Hindi.

[📖 Live Demo](https://vikas972.github.io/genAINews/) · [📋 Configuration Guide](https://vikas972.github.io/genAINews/configuration) · [हिंदी](README_hi.md)

</div>

## Screenshots

<table>
<tr>
<td width="50%">
<p align="center"><strong>Daily Overview</strong></p>
<img src="docs/assets/overview_en.png" alt="Daily Overview" />
</td>
<td width="50%">
<p align="center"><strong>News Detail</strong></p>
<img src="docs/assets/one_news_en.png" alt="News Detail" />
</td>
</tr>
</table>

<details>
<summary><strong>Terminal Output</strong></summary>
<br>
<p align="center">
  <img src="docs/assets/terminal_log.png" alt="Terminal Output" width="400" />
</p>
</details>

## Features

- **📡 Multi-Source Aggregation** — Collects from Hacker News, RSS feeds, Reddit, Telegram channels, and GitHub (releases & user events)
- **🤖 AI-Powered Scoring** — Uses Claude, GPT-4, Gemini, DeepSeek, Doubao, MiniMax, or any OpenAI-compatible API to rate each item 0-10, filtering out the noise
- **🌐 Bilingual Summaries** — Generates daily reports in English and/or Hindi (`hi` in config)
- **🔍 Content Enrichment** — Searches the web to provide background knowledge for unfamiliar concepts
- **💬 Community Voices** — Collects and summarizes discussions from comments on HackerNews, Reddit, etc.
- **🔗 Cross-Source Deduplication** — Merges duplicate items from different platforms automatically
- **📧 Email Subscription** — Self-hosted newsletter system (SMTP/IMAP) that handles "Subscribe" requests automatically
- **📝 Static Site Generation** — Deploys as a GitHub Pages site via GitHub Actions, updated on a schedule
- **⚙️ Fully Configurable** — Single JSON config file, easy to customize sources, thresholds, and AI providers
- **🧙 Setup Wizard** — Interactive CLI that recommends sources based on your interests, with a [curated preset library](https://vikas972.github.io/genAINews/presets) open to community contributions

## How It Works

```
              ┌──────────┐
              │ Hacker   │
┌─────────┐   │ News     │   ┌──────────┐   ┌──────────┐   ┌──────────┐
│  RSS    │──▶│ Reddit   │──▶│ AI Score │──▶│ Enrich   │──▶│ Summary  │
│ Telegram│   │ GitHub   │   │ & Filter │   │ & Search │   │ & Deploy │
└─────────┘   └──────────┘   └──────────┘   └──────────┘   └──────────┘
  Fetch from      Merge &        Score          Web search     Generate
  all sources    deduplicate     0-10 each      background     Markdown &
                                & filter        knowledge      deploy site
```

1. **Fetch** — Pull latest content from all configured sources concurrently
2. **Deduplicate** — Merge items pointing to the same URL across different platforms
3. **Score** — AI rates each item 0-10 based on technical depth, novelty, and impact
4. **Filter** — Keep only items above your configured threshold (default: 6.0)
5. **Enrich** — For high-scoring items, search the web for background context and collect community discussions
6. **Summarize** — Generate a structured Markdown report with summaries, tags, and references
7. **Deploy** — Optionally publish to GitHub Pages as a daily-updated static site

## Quick Start

### 1. Install

#### Option A: From PyPI (quickest)

Install [from PyPI](https://pypi.org/project/genaiforgenews/) and use a dedicated working directory (the app reads `./data/` and `.env` from your **current folder**):

```bash
pip install genaiforgenews
mkdir ~/my-genaiforgenews && cd ~/my-genaiforgenews
# Add API keys to .env (see .env.example in the repo)
genaiforgenews-wizard   # writes data/config.json
genaiforgenews
```

Full notes for pip users and **maintainer publishing steps** (build, TestPyPI, Twine) are in [`docs/PYPI.md`](docs/PYPI.md).

#### Option B: Local installation (git clone)

```bash
git clone https://github.com/vikas972/genAINews.git
cd genAINews

# Install with uv (recommended)
uv sync

# Or with pip (editable)
pip install -e .
```

#### Option C: Docker

```bash
git clone https://github.com/vikas972/genAINews.git
cd genAINews

# Configure environment
cp .env.example .env
cp data/config.example.json data/config.json
# Edit .env and data/config.json with your API keys and preferences

# Run with Docker Compose
docker compose run --rm genaiforgenews

# Or run with custom time window
docker compose run --rm genaiforgenews --hours 48
```

### 2. Configure

**Option A: Interactive wizard (recommended)**

```bash
uv run genaiforgenews-wizard
```

The wizard asks about your interests (e.g. "LLM inference", "嵌入式", "web security") and auto-generates `data/config.json` from a [curated preset library](https://vikas972.github.io/genAINews/presets) + optional AI recommendations.

**Option B: Manual configuration**

```bash
cp .env.example .env          # Add your API keys
cp data/config.example.json data/config.json  # Customize your sources
```

Here's what a config looks like:

```jsonc
{
  "ai": {
    "provider": "openai",       // or "anthropic", "gemini", "doubao", "minimax"
    "model": "gpt-4",
    "api_key_env": "OPENAI_API_KEY",
    "languages": ["en", "hi"]   // English + Hindi output
  },
  "sources": {
    "hackernews": { "enabled": true, "fetch_top_stories": 20, "min_score": 100 },
    "rss": [
      { "name": "Simon Willison", "url": "https://simonwillison.net/atom/everything/" }
    ],
    "reddit": {
      "subreddits": [{ "subreddit": "MachineLearning", "sort": "hot" }],
      "fetch_comments": 5
    },
    "telegram": {
      "channels": [{ "channel": "zaihuapd", "fetch_limit": 20 }]
    }
  },
  "filtering": {
    "ai_score_threshold": 6.0,
    "time_window_hours": 24
  }
}
```

For the full reference, see the [Configuration Guide](docs/configuration.md).

### 3. Run

#### Local Installation

```bash
uv run genaiforgenews           # Run with default 24h window
uv run genaiforgenews --hours 48  # Fetch from last 48 hours
```

#### With Docker

```bash
docker compose run --rm genaiforgenews           # Run with default 24h window
docker compose run --rm genaiforgenews --hours 48  # Fetch from last 48 hours
```

The generated report will be saved to `data/summaries/`.

### 4. Automate (Optional)

genaiforgenews works great as a **GitHub Actions** cron job. See [`.github/workflows/daily-summary.yml`](.github/workflows/daily-summary.yml) for a ready-to-use workflow that generates and deploys your daily briefing to GitHub Pages automatically.

## Supported Sources

| Source | What it fetches | Comments |
|--------|----------------|----------|
| **Hacker News** | Top stories by score | Yes (top N comments) |
| **RSS / Atom** | Any RSS or Atom feed | — |
| **Reddit** | Subreddits + user posts | Yes (top N comments) |
| **Telegram** | Public channel messages | — |
| **GitHub** | User events & repo releases | — |

## MCP Integration

genaiforgenews ships with a built-in [MCP](https://modelcontextprotocol.io/) server so AI assistants can drive the pipeline programmatically.

```bash
# Start the MCP server (stdio mode)
uv run genaiforgenews-mcp
```

Available tools include `hz_validate_config`, `hz_fetch_items`, `hz_score_items`, `hz_filter_items`, `hz_enrich_items`, `hz_generate_summary`, and `hz_run_pipeline`.

See [`src/mcp/README.md`](src/mcp/README.md) for the full tool reference and [`src/mcp/integration.md`](src/mcp/integration.md) for client setup.

## Roadmap

- [x] Multi-source aggregation (HN, RSS, Reddit, Telegram, GitHub)
- [x] AI scoring with multiple providers
- [x] Bilingual summary generation (EN/HI)
- [x] Web search for background enrichment
- [x] Community discussion collection
- [x] GitHub Pages deployment
- [x] **Email Subscription** (SMTP/IMAP automated newsletter)
- [x] **Docker deployment support**
- [x] **MCP server integration**
- [x] Web UI dashboard
- [x] **Setup Wizard** — interactive CLI that recommends sources based on user interests
- [ ] **Improved Web UI** — better digest and article detail experience
- [ ] Slack / Webhook notification
- [ ] More source types (Twitter/X, Discord, etc.)
- [ ] Custom scoring prompts per source

## Contributing

Contributions are welcome! Feel free to open issues or submit pull requests.

### Contribute Source Presets

genaiforgenews's setup wizard uses a community-maintained [preset library](https://vikas972.github.io/genAINews/presets) to recommend sources. **We'd love your help expanding it!**

1. Fork this repo
2. Add your sources to `data/presets.json` (provide both English and Hindi `description` / `description_hi` where useful)
3. Submit a PR

Great candidates: niche RSS feeds, active subreddits, notable GitHub accounts, or Telegram channels in your area of expertise.

## Acknowledgements

- Special thanks to [LINUX.DO](https://linux.do/) for providing a promotion platform.
- Special thanks to [HelloGitHub](https://hellogithub.com/) for valuable guidance and suggestions.

## License

[MIT](LICENSE)
