DOES-Applications/bls-data
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix broken series-ID builders; env-var config; project README

Browse Source 
Repair every dead series-ID encoding (library now 82/82 query helpers
return live data, verified against the BLS API):

- JOLTS: 21-char format (was 18) — add state/area/sizeclass fields
- OES: national area code 0000000 (was invalid 0000400)
- ECI: correct owner/component/estimate encoding, default unadjusted (CIU)
- Productivity: 4-digit sector + 4-digit measure codes (was 2+3)
- QCEW: 13-char timeseries-API form (ENUUS00010510 / ENU{fips}00010{own}10)
- PPI: repoint finished-goods -> final demand (WPUFD4); keep alias
- ECEC: drop fabricated health-insurance/retirement helpers; add total benefits
- wages SOC: software developers 151132 -> 151252 (2018 SOC)

Tooling/docs:
- config reads BLS_API_KEY env var (takes precedence; config.py gitignored)
- add requirements.txt
- rewrite README as project front door + coverage table + limitations
- correct JOLTS/OES tables in series_id_formats.md, USAGE.md, dataset explorer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
David Boyd
2026-06-22 09:45:27 -04:00
parent 20bd3e9a2f
commit 4fe734381e
10 changed files with 246 additions and 245 deletions
Download Patch File Download Diff File Expand all files Collapse all files

294
README.md
View File

@ -1,16 +1,117 @@
# BLS Data Reference
# BLS Data Library
Bureau of Labor Statistics — API access, dataset catalog, series ID formats, and field schemas.
A small, dependency-light Python library for pulling U.S. Bureau of Labor Statistics
(BLS) time series — unemployment, payrolls, inflation, wages, job openings, and
productivity — with pre-built series-ID helpers so you never have to hand-encode a
series ID.
## Quick Start
```python
from bls_client import BLSClient
from bls_client.queries import employment, prices
1. Register for an API key: https://data.bls.gov/registrationEngine/
2. Base API endpoint: `https://api.bls.gov/publicAPI/v2/timeseries/data/`
3. Content-Type: `application/json` (POST)
client = BLSClient(API_KEY)
# Latest national nonfarm payrolls
client.fetch_latest(employment.nonfarm_payrolls(), years=1)
# A labeled CPI dashboard in one call
client.fetch_named(prices.cpi_dashboard(), 2024, 2025)
```
All 82 zero-argument query helpers are verified against the live API.
---
## API Versions
## Install
```bash
pip install -r requirements.txt # just `requests`
```
## Configure your API key
Register for a free key (instant, no approval): https://data.bls.gov/registrationEngine/
The free v2 key allows 500 queries/day, 50 series/query, 20 years/query.
```bash
cp config.example.py config.py
export BLS_API_KEY="your-key" # preferred — config.py reads this env var
```
`config.py` is gitignored; the env var takes precedence over anything written in the file.
You can also pass the key directly: `BLSClient("your-key")`.
## Quick start
```bash
python3 examples/basic_pull.py # payrolls, CPI dashboard, DC-region unemployment
python3 examples/custom_series.py # building custom series IDs
```
---
## What's in the box
```
bls_client/
├── client.py BLSClient — batching, named fetches, catalog metadata, row flattening
├── series.py low-level series-ID builders (LAUS, CES, CPI, PPI, OES, JOLTS, ECI, QCEW, productivity)
└── queries/
├── employment.py payrolls, unemployment (LAUS), JOLTS, QCEW
├── prices.py CPI, PPI, average prices, import/export prices
├── wages.py OES occupational wages, ECI, ECEC
└── productivity.py major-sector productivity & costs
```
`BLSClient` highlights:
- `fetch(ids, start, end)` — auto-batches >50 series, returns `{series_id: {data, catalog}}`
- `fetch_latest(ids, years=N)` — most recent N years
- `fetch_named({label: id})` — returns results keyed by your labels
- `latest_obs(series)` / `to_rows(results)` — convenience for the most recent value / CSV-ready rows
See **[USAGE.md](USAGE.md)** for the full API and **[series_id_formats.md](series_id_formats.md)**
for the series-ID decode tables.
---
## Coverage
| Survey | Helpers | Notes |
|---|---|---|
| LAUS — local area unemployment | state / metro / county rates, DC-region dashboard | ✅ |
| CES — payroll employment | national by supersector; state/metro | ✅ |
| CPS — household survey | national unemployment rate, participation | ✅ |
| JOLTS — job openings & turnover | openings/hires/quits/layoffs, dashboard | ✅ |
| CPI — consumer prices | all-items, core, food, energy, gasoline, …; dashboard | ✅ |
| PPI — producer prices | all commodities, final demand, food, energy | ✅ |
| OES — occupational wages | employment + wage percentiles by SOC; 15 common occupations | ✅ |
| ECI — employment cost index | total comp / wages / benefits × civilian/private/gov | ✅ |
| ECEC — employer cost levels | total compensation, total benefits | ✅ (totals only) |
| Productivity & costs | output/hr, ULC, comp, hours × business/nonfarm/manufacturing | ✅ |
| QCEW — quarterly census | national & state private totals | ⚠️ totals only via this API |
## Known limitations / what "complete" would add
- **QCEW** is only partially served by the BLS *timeseries* API used here (national and
state-level totals work). County- and industry-level QCEW detail requires the separate
**QCEW Open Data API** (CSV: `https://data.bls.gov/cew/data/api/...`). Not yet wired up.
- **ECEC benefit subcomponents** (health insurance, retirement & savings, etc.) need
specific benefit-subcell codes from the ECEC component list; only the compensation and
benefits totals are currently exposed.
- **No caching / rate-limit handling.** Repeated runs spend against the 500/day quota; a
small on-disk cache and a friendly error on `REQUEST_NOT_PROCESSED` (quota hit) would help.
- **No packaging.** Importable in-tree but not `pip install`-able; add a `pyproject.toml`
to ship it as a real package.
- **No automated tests.** A pytest suite asserting builder outputs against known-good IDs
(and a live smoke test behind a marker) would lock the series-ID encodings in place.
---
## Appendix: BLS API reference
Reference material for working directly with the API (the library wraps all of this).
### API versions
| Feature | v1 (no key) | v2 (registered) |
|---------------------------|-------------|-----------------|
@ -21,174 +122,45 @@ Bureau of Labor Statistics — API access, dataset catalog, series ID formats, a
| Series descriptions | No | Yes |
| Calculations | No | Yes |
---
### Endpoints
## Registration
- URL: https://data.bls.gov/registrationEngine/
- Free, no approval needed — instant key via email
- Key goes in the JSON payload as `"registrationkey": "YOUR_KEY"`
---
## Endpoints
### GET: Survey List
```
GET https://api.bls.gov/publicAPI/v2/surveys
GET https://api.bls.gov/publicAPI/v2/surveys/{survey_abbreviation}
```
Returns all survey codes and names. See `surveys.json` for full list.
### GET: Popular Series
```
GET https://api.bls.gov/publicAPI/v2/timeseries/popular
GET https://api.bls.gov/publicAPI/v2/timeseries/popular?survey={abbreviation}
```
Returns the 25 most-requested series IDs for a survey.
### POST: Time Series Data
```
POST https://api.bls.gov/publicAPI/v2/timeseries/data/
Content-Type: application/json
GET /v2/surveys # all survey codes and names (see surveys.json)
GET /v2/surveys/{abbr} # one survey
GET /v2/timeseries/popular?survey={abbr} # 25 most-requested series IDs for a survey
POST /v2/timeseries/data/ # the time-series data endpoint
```
**Minimal request (unregistered):**
Base URL: `https://api.bls.gov/publicAPI/v2`
**POST body (registered, all v2 features):**
```json
{
"seriesid": ["LAUST110000000000003", "CES0000000001"],
"startyear": "2023",
"endyear": "2025"
}
```
**Full request (registered, v2 features):**
```json
{
"seriesid": ["LAUST110000000000003", "CES0000000001"],
"startyear": "2020",
"endyear": "2025",
"startyear": "2020", "endyear": "2025",
"registrationkey": "YOUR_KEY",
"catalog": true,
"calculations": true,
"annualaverage": true,
"aspects": true
"catalog": true, "calculations": true, "annualaverage": true
}
```
**Response schema:**
```json
{
"status": "REQUEST_SUCCEEDED",
"responseTime": 114,
"message": [],
"Results": {
"series": [
{
"seriesID": "LAUST110000000000003",
"catalog": {
"series_title": "...",
"survey_name": "...",
"measure_data_type": "..."
},
"data": [
{
"year": "2025",
"period": "M12",
"periodName": "December",
"value": "6.4",
"footnotes": [
{ "code": "R", "text": "Data were subject to revision on April 8, 2026." }
],
"calculations": {
"net_changes": { "1": "0.1", "3": "-0.2", "6": "0.5", "12": "-0.3" },
"pct_changes": { "1": "1.6", "3": "-3.0", "6": "8.3", "12": "-4.5" }
}
}
]
}
]
}
}
```
**Period codes:** monthly `M01`–`M12` (`M13` = annual avg); quarterly `Q01`–`Q04` (`Q05` = annual avg); annual `A01`.
**Status codes:** `REQUEST_SUCCEEDED`, `REQUEST_FAILED`, `REQUEST_NOT_PROCESSED` (often = daily quota hit).
**Footnote codes:** `R` revised, `P` preliminary, `X`/`N` unavailable.
**Period codes:**
- Monthly: `M01`–`M12`, `M13` (annual average)
- Quarterly: `Q01`–`Q04`, `Q05` (annual average)
- Annual: `A01`
A full real response is saved in `api_response_example.json`.
**Status codes:** `REQUEST_SUCCEEDED`, `REQUEST_FAILED`, `REQUEST_NOT_PROCESSED`
### Bulk flat files
**Error footnote codes:**
- `R` — Revised
- `P` — Preliminary
- `X` — Data unavailable (e.g., government shutdown gap)
- `N` — Not available
Base: `https://download.bls.gov/pub/time.series/` — each survey folder has
`{prefix}.series` (master list), `{prefix}.data.*` (observations), and `{prefix}.{dimension}`
decode tables (area, industry, measure). Tab-delimited; handy for bulk PostgreSQL ingest.
Key prefixes: `la/` LAUS, `ce/` CES, `sm/` state-metro, `en/` QCEW, `oe/` OES, `jt/` JOLTS,
`cu/` CPI-U, `wp/` PPI.
---
### Other files in this repo
## Python Example
```python
import requests
API_KEY = "YOUR_KEY"
BASE_URL = "https://api.bls.gov/publicAPI/v2/timeseries/data/"
def get_series(series_ids, start_year, end_year):
payload = {
"seriesid": series_ids,
"startyear": str(start_year),
"endyear": str(end_year),
"registrationkey": API_KEY,
"catalog": True,
"calculations": True,
"annualaverage": True,
}
r = requests.post(BASE_URL, json=payload)
r.raise_for_status()
data = r.json()
if data["status"] != "REQUEST_SUCCEEDED":
raise ValueError(f"BLS API error: {data['message']}")
return data["Results"]["series"]
# DC unemployment rate (LAUS)
series = get_series(["LAUST110000000000003"], 2020, 2025)
for obs in series[0]["data"]:
print(obs["year"], obs["periodName"], obs["value"])
```
---
## Bulk Download (flat files)
Base URL: `https://download.bls.gov/pub/time.series/`
Each survey folder contains:
- `{prefix}.series` — master list of all series IDs with metadata
- `{prefix}.data.{N}.{name}` — actual observations, split by category
- `{prefix}.{dimension}` — lookup/decode tables (area, industry, measure, etc.)
Key folder prefixes:
```
la/ — LAUS (local area unemployment)
ce/ — CES national employment
sm/ — State & Metro employment (CES state)
en/ — QCEW
oe/ — OES (occupational employment)
jt/ — JOLTS
cu/ — CPI-U
wp/ — PPI commodities
```
Files are tab-delimited. Useful for bulk PostgreSQL ingest.
---
## Files in this folder
- `README.md` — this file
- `series_id_formats.md` — series-ID decode tables for each survey
- `qcew_field_schema.md` — QCEW quarterly/annual CSV field layouts
- `surveys.json` — complete survey list from the API
- `series_id_formats.md` — series ID decode tables for each key dataset
- `qcew_field_schema.md` — QCEW quarterly and annual CSV field layouts
- `api_response_example.json` — real API response sample
- `bls_dataset_explorer.py` / `.html` — browsable survey catalog
- `dc_md_va_unemployment.py` / `generate_report.py` — example report generators