Metadata-Version: 2.4
Name: indeed-scraper-api
Version: 0.0.1
Summary: Python client for scraping Indeed job listings using the ScrapingBee web scraping API
Author: wordstotech
License: MIT
Project-URL: Homepage, https://www.scrapingbee.com/blog/how-to-scrape-indeed/
Keywords: indeed scraper,indeed job scraper,indeed web scraper,best indeed scraper,how to scrape indeed,scrapingbee
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests
Dynamic: license-file

# Indeed Scraper API (Python)

A small Python client that turns Indeed job pages into clean JSON using the [ScrapingBee web scraping API](https://www.scrapingbee.com/features/ai-web-scraping-api/). You pass a search query and a location, and you get back a list of jobs with title, company, location, and link. The headless browser rendering, residential proxies, and HTML parsing are handled for you.

Indeed renders its results with JavaScript and defends against automated traffic, so a plain `requests.get` returns little. This [Indeed scraper](https://www.scrapingbee.com/scrapers/indeed-api/) sends each request through ScrapingBee with JavaScript rendering and residential proxies switched on, then applies the extraction rules server-side so you never touch BeautifulSoup.

## Install

```bash
pip install indeed-scraper-api
```

## Quickstart

```python
from indeed_scraper_api import IndeedScraper

scraper = IndeedScraper(api_key="YOUR_SCRAPINGBEE_API_KEY")

result = scraper.search("python developer", location="New York", start=0)
for job in result["jobs"]:
    print(job["title"], "-", job["company"], "-", job["location"])
```

Get an API key with 1,000 free credits at [ScrapingBee](https://www.scrapingbee.com/); no card required.

## What you get back

`search()` returns a dictionary with a `jobs` list, one entry per Indeed job card:

```json
{
  "jobs": [
    {
      "title": "Python Developer",
      "company": "Acme Corp",
      "location": "New York, NY",
      "url": "/rc/clk?jk=..."
    }
  ]
}
```

The fields come straight from the CSS selectors documented in ScrapingBee's [data extraction rules](https://www.scrapingbee.com/documentation/data-extraction/), so you can add your own fields by passing a custom `extract_rules` dictionary.

## API reference

**`IndeedScraper(api_key, timeout=60)`** creates a client.

- **`search(query, location="", start=0, **options)`** returns structured jobs for one results page. Indeed pages in steps of 10, so `start=10` is page two.
- **`pages(query, location="", count=3, **options)`** yields structured results for the first `count` pages.
- **`scrape(url, extract_rules=None, **options)`** fetches any Indeed URL. Returns rendered HTML, or JSON when you pass `extract_rules`.

Every method accepts these overrides:

| Option | Default | Purpose |
|---|---|---|
| `render_js` | `True` | Run the page in a headless browser |
| `premium_proxy` | `True` | Route through residential proxies |
| `country_code` | `"us"` | Proxy country |
| `wait` | `5000` | Milliseconds to wait after load |
| `extract_rules` | `JOB_RULES` | Override the returned fields |

## Credit cost

With residential proxies and JavaScript rendering both on, each request costs 25 ScrapingBee credits (Premium tier plus JS). Drop `premium_proxy` for cheaper datacenter requests where a target allows it. See [ScrapingBee pricing](https://www.scrapingbee.com/pricing) for the credit table.

## Requirements

Python 3.8 or newer, the `requests` library (installed automatically), and a ScrapingBee API key. This client is an independent wrapper built on the public ScrapingBee API and is not affiliated with Indeed.

## Links

- [How to scrape Indeed (ScrapingBee guide)](https://www.scrapingbee.com/blog/how-to-scrape-indeed/)
- [ScrapingBee Indeed scraper](https://www.scrapingbee.com/scrapers/indeed-api/)
- [Data extraction rules](https://www.scrapingbee.com/documentation/data-extraction/)

## License

MIT
