Metadata-Version: 2.4
Name: billboard-charts
Version: 0.2.0
Summary: Billboard chart scraping that refuses to invent history
Author: Prangon Barua
License: MIT License
        
        Copyright (c) 2026 Prangon Barua
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR OTHER DEALINGS IN THE SOFTWARE.
        
        ---
        
        The MIT terms above cover this repository's source code. The chart data under
        data/ is compiled from Billboard's published weekly charts; Billboard retains
        whatever rights it holds in the underlying chart rankings, and this project
        claims none of them.
License-File: LICENSE
Keywords: billboard,charts,hot-100,music,scraper
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Requires-Python: >=3.9
Requires-Dist: beautifulsoup4>=4.9
Requires-Dist: pandas>=1.3
Requires-Dist: requests>=2.25
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Description-Content-Type: text/markdown

# billboard-charts

Billboard chart scraping that refuses to invent history.

Billboard.com answers **any** date you give it. Ask for a week a chart never
published and you get HTTP 200, valid markup, and a plausible row count, but
the rankings belong to a different week, stamped with the date you asked for.
Store that and you have fabricated chart history that looks complete.

This library treats the page's own `Week of ...` heading as the authority and
rejects anything that does not match the week you requested.

Every chart on billboard.com works. There is no allowlist: pass the slug from
the chart's URL and you get that chart.

## Installation

```bash
pip install billboard-charts
```

## Quickstart

```
>>> from billboard_charts import fetch_chart
>>> rows = fetch_chart('hot-100', '2000-10-21')
>>> len(rows)
100
>>> rows[0]
{'Date': '2000-10-21', 'Rank': 1, 'Song': 'Come On Over Baby (All I Want Is You)', 'Artist': 'Christina Aguilera', 'Last Week': 1, 'Peak Position': 1, 'Weeks on Chart': 12}
>>> rows[1]['Song'], rows[1]['Artist']
('Music', 'Madonna')
>>> rows[99]
{'Date': '2000-10-21', 'Rank': 100, 'Song': 'Lucky', 'Artist': 'Britney Spears', 'Last Week': 86, 'Peak Position': 23, 'Weeks on Chart': 11}
```

Omit the date and you get the current week:

```
>>> latest = fetch_chart('hot-100')
>>> latest[0]['Date'], latest[0]['Song']
('2026-08-01', "Choosin' Texas")
```

Any slug, not just the Hot 100:

```
>>> bu = fetch_chart('bubbling-under-hot-100-singles', '2016-04-30')
>>> len(bu)
25
>>> bu[0]
{'Date': '2016-04-30', 'Rank': 1, 'Song': 'I Hate U I Love U', 'Artist': "Gnash Featuring Olivia O'Brien", 'Last Week': '-', 'Peak Position': 1, 'Weeks on Chart': 2}
```

Now the part that separates this from a plain scraper. The Hot 100 dates to
Saturdays; ask for the Sunday and Billboard hands back the following week
rather than an error:

```
>>> fetch_chart('hot-100', '2000-10-22')
Traceback (most recent call last):
  ...
billboard_charts.scraper.WeekNotPublished: hot-100 2000-10-22: Billboard served 2000-10-28 instead
```

Ask for a date before a chart existed and Billboard clamps to the chart's
first week:

```
>>> fetch_chart('hot-dance-singles-sales', '1974-01-05')
Traceback (most recent call last):
  ...
billboard_charts.scraper.WeekNotPublished: hot-dance-singles-sales 1974-01-05: Billboard served 1985-01-19 instead
```

Ask for a slug that is not a weekly chart and Billboard redirects to the
year-end page, which carries no week at all:

```
>>> fetch_chart('hot-100-songs', '2016-04-30')
Traceback (most recent call last):
  ...
billboard_charts.scraper.WeekNotPublished: hot-100-songs 2016-04-30: page carries no chart date (redirect, unpublished week, or markup change)
```

All three responses are HTTP 200 with well-formed markup and a full row count.
Nothing but the date on the page tells them apart.

## Guide

### fetch_chart

```python
fetch_chart(slug, date=None, *, session=None, timeout=15)
```

Fetches one chart week and returns a list of entry dicts ordered by rank.

* `slug` (str): the chart slug from its billboard.com URL. For
  `https://www.billboard.com/charts/hot-100/` the slug is `hot-100`. Other
  examples: `billboard-200`, `bubbling-under-hot-100-singles`,
  `hot-adult-r-and-b-airplay`, `hot-dance-singles-sales`.
* `date` (str or None): the chart week as `'YYYY-MM-DD'`. Must be the exact
  date Billboard publishes the chart under, which is a Saturday for most
  charts. `None` fetches the current week and skips the served-week check,
  since there is no requested date to compare against. A string in any other
  format raises `ValueError` before any request is made.
* `session` (requests.Session or None): reuse one session across many calls.
  One is created per call if omitted, which is wasteful in a loop.
* `timeout` (int): per-request timeout in seconds. Default 15.

Raises `WeekNotPublished` if the page names a week other than the one
requested, or names no week at all. Raises `requests.HTTPError` on a non-200
response, which is what an unknown slug produces (404).

### Entry attributes

Every entry is a plain `dict` with these seven keys, in this order. The same
list is exported as `billboard_charts.COLUMNS`, and it is the CSV header that
`backfill` writes.

* `Date` (str): the chart week as `'YYYY-MM-DD'`. Identical for every entry in
  one response. When you pass a date, this is that date, guaranteed to match
  the week the page served. When you omit it, this is the week the page
  reported.
* `Rank` (int): this week's position, starting at 1.
* `Song` (str): the title as printed. For an albums chart such as
  `billboard-200` this holds the album title.
* `Artist` (str): the credited artist, with runs of whitespace collapsed to
  single spaces, so features and collaborations read as one line
  (`"Gnash Featuring Olivia O'Brien"`). Falls back to `'Unknown'` when the
  artist element is missing from the row.
* `Last Week` (int or str): last week's rank. The string `'-'` for a debut,
  for a re-entry, and for any row where Billboard prints no previous position.
  This is the one key whose type varies, so compare against `'-'` before doing
  arithmetic on it.
* `Peak Position` (int): the highest rank the title has reached. Falls back to
  the entry's own `Rank` when the page omits the stat, which is correct for a
  debut and is why this key is never `'-'`.
* `Weeks on Chart` (int): total weeks charted including this one. Falls back
  to `1` when the page omits the stat.

Older charts carry fewer rows than their name implies. The Bubbling Under Hot
100 Singles chart above returns 25 entries, and the Hot Dance Singles Sales
chart ran at 50 or fewer for most of its life. Do not use row count to
validate a response.

### Exceptions

* `BillboardError`: base class for every scrape failure this library raises.
* `WeekNotPublished(BillboardError)`: the response is not the week that was
  requested. Its message names both dates when the page carried a week, and
  says `page carries no chart date` when it did not. This is authoritative
  rather than transient, so it is never worth retrying.

### backfill

```python
backfill(slug, csv_path, first_week, last_week=None, *,
         session=None, fetch=fetch_chart, on_progress=None,
         checkpoint_every=25, max_attempts=3)
```

Fetches every missing week of a chart into a CSV and returns a
`BackfillResult`.

```
>>> from billboard_charts import backfill
>>> result = backfill('hot-dance-singles-sales', 'dance.csv',
...                   '2007-01-27', '2007-02-24')
>>> print(result)
hot-dance-singles-sales: 5 weeks, 2007-01-27 -> 2007-02-24, 5 written, 0 failed, 0 clamped
>>> result.written
['2007-01-27', '2007-02-03', '2007-02-10', '2007-02-17', '2007-02-24']
```

* `slug` (str): as for `fetch_chart`.
* `csv_path` (str or Path): the CSV to create or extend. An existing file is
  read first and only the weeks it lacks are fetched, so an interrupted run
  loses nothing and a re-run costs only the gaps.
* `first_week` (str): first chart week to request, `'YYYY-MM-DD'`.
* `last_week` (str or None): last chart week. Omit only for a chart still
  running. Always pass one for a discontinued chart, otherwise every date
  after its final week is still requested and rejected one round trip at a
  time.
* `session`, `fetch`: a session to reuse, and the fetch function to call.
  `fetch` exists so tests can inject a fake.
* `on_progress` (callable or None): called as `on_progress(week, status)`
  after each week, where status is `'written'`, `'clamped'`, or `'failed'`.
* `checkpoint_every` (int): write the CSV to disk every N weeks. Default 25.
* `max_attempts` (int): attempts per week for transient errors, with a backoff
  of 3, 6, ... seconds between them. `WeekNotPublished` is never retried.

The CSV is deduplicated on `(Date, Rank)` and sorted by date then rank on
every save, so the file stays canonical no matter what order weeks arrived in.

### BackfillResult

A dataclass describing the run. `str(result)` gives the one-line summary shown
above.

* `slug` (str): the chart that was fetched.
* `weeks` (int): distinct weeks in the CSV after the run, existing rows
  included, not just the ones this run added.
* `written` (list of str): weeks fetched and stored this run.
* `clamped` (list of str): weeks rejected by the clamp guard below.
* `failed` (list of str): weeks that raised `WeekNotPublished` or exhausted
  `max_attempts`. Most entries here are genuine gaps in the chart's history.
* `first`, `last` (str or None): earliest and latest dates in the CSV.

### weeks_between

```python
weeks_between(first, last=None)
```

Every weekly chart date from `first` through `last` inclusive, stepping seven
days. Without `last`, it runs through the current week.

```
>>> from billboard_charts import weeks_between
>>> weeks_between('2000-10-21', '2000-11-11')
['2000-10-21', '2000-10-28', '2000-11-04', '2000-11-11']
>>> len(weeks_between('1958-08-04'))
3549
```

This is the calendar the backfill walks. It is a naive seven-day sequence, not
a claim that all of those weeks were published.

### Other exports

* `parse_chart(html, slug, date)`: the parser `fetch_chart` wraps. Takes chart
  HTML and returns entries, applying the same served-week check, so you can
  run it against saved pages with no network.
* `make_session(headers=None)`: a `requests.Session` with a browser
  User-Agent. Billboard returns 403 to the default requests User-Agent, so
  pass this rather than a bare session.
* `COLUMNS`: the seven entry keys, in order.

### Command line

```bash
billboard-backfill <slug> <csv_path> <first_week> [last_week] [-q]
```

```
$ billboard-backfill hot-dance-singles-sales dance.csv 2007-02-03 2007-02-24
  2007-02-03 written
  2007-02-10 written
  2007-02-17 written
  2007-02-24 written
hot-dance-singles-sales: 4 weeks, 2007-02-03 -> 2007-02-24, 4 written, 0 failed, 0 clamped
```

```
$ head -3 dance.csv
Date,Rank,Song,Artist,Last Week,Peak Position,Weeks on Chart
2007-02-03,1,Jump,Madonna,1,1,12
2007-02-03,2,Every Day Is Exactly The Same,Nine Inch Nails,2,1,42
```

`-q` suppresses the per-week lines and prints only the final summary. Failed
and clamped weeks are listed after the summary, up to the first 20 of each.

## Notes

### The two guards

**Served-week check.** A slug that is not a weekly chart redirects to
`/charts/year-end/<slug>/`, and a date outside a chart's run is clamped to the
nearest published week. Both return 200 with clean markup and a passing row
count, so only the date on the page distinguishes them. A mismatch raises
`WeekNotPublished`. This runs on every `fetch_chart` call that names a date.

**Clamp guard.** Some clamped responses still carry the requested date, which
defeats the check above. Those are caught during a backfill by comparing each
week's full `(rank, song, artist)` ordering against its chronological
predecessor. Real consecutive chart weeks always differ somewhere in 100 rows;
an identical ordering means the same week was served twice.

The predecessor is looked up per week, not carried in a rolling variable. That
distinction matters on a re-run over a partly filled CSV, where the missing
weeks are scattered: a rolling signature compares each one against whatever
week happened to be written last, and fabricated weeks slip through. A
duplicate `(Date, Rank)` check will not catch it either, because the dates
differ and only the ranking repeats.

### Gaps are real

Charts are not contiguous. Billboard skipped a week each New Year for years on
some charts, and others have genuine mid-year holes. A backfill that returns
fewer weeks than the calendar suggests is usually correct, and the weeks in
`result.failed` are the evidence, not an error to paper over.

Verify a gap rather than assuming it. An unpublished week's page has no
`Week of` heading. An empty page alone is not proof: Billboard served empty
pages for seven real weeks on 2026-08-02 that returned full data a day later.

### Being a good citizen

Every call is one HTTP request to billboard.com with no caching. A full
history is thousands of requests, so pass a `session`, run backfills once and
keep the CSV, and do not parallelize a backfill across processes.

## Dependencies

* Python 3.9 or newer
* [requests](https://pypi.org/project/requests/) 2.25 or newer
* [beautifulsoup4](https://pypi.org/project/beautifulsoup4/) 4.9 or newer
* [pandas](https://pypi.org/project/pandas/) 1.3 or newer, used by `backfill`
  for the resumable CSV

Development extras (`pip install -e '.[dev]'`) add pytest. Run the suite with
`pytest`; it parses inline HTML and drives the backfill through a fake fetch
that reproduces Billboard's clamping, so no test touches the network.

## License

MIT. See [LICENSE](LICENSE).
