Add QCEW CSV module, ECEC breakdown, response caching + typed errors

Item 1 — QCEW county/industry detail:
- new bls_client/qcew.py: thin client over the QCEW Open Data CSV service
  (area/industry/size slices; no API key, no quota), with filter_rows helper
- covers the county/industry detail the timeseries API can't reach

Item 2 — ECEC benefit breakdown:
- new series.ecec() builder from the authoritative cm.estimate decode table
- real helpers: wages, total benefits, paid leave, supplemental pay,
  health insurance, retirement & savings, legally required + ecec_dashboard()
- FIX: ecec_total_benefits was CMU1036... (education/health industries only);
  correct all-civilian total benefits is CMU1030000000000D

Item 3 — caching + typed errors:
- bls_client/cache.py FileCache (on-disk, TTL); BLSClient(cache=True),
  queries_used counter (cache hits don't spend quota)
- bls_client/errors.py: BLSError / BLSQuotaError / BLSRequestError; quota
  exhaustion now raises a clear BLSQuotaError instead of a generic RuntimeError

Tests: 119 offline (+24) incl. ECEC goldens, QCEW fixture parse, cache/quota
behavior; live smoke extended to ECEC + QCEW. Helper sweep: 96/96 live.
README: caching/QCEW usage, updated coverage + limitations.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-22 10:38:12 -04:00
parent 881d62cf32
commit 9e2c55c568
14 changed files with 499 additions and 44 deletions

74
bls_client/qcew.py Normal file
View File

@ -0,0 +1,74 @@
"""QCEW Open Data Access — county/industry employment & wages.
The BLS *timeseries* API only carries QCEW national/state totals. The full
QCEW detail (every county, every NAICS industry, establishment counts and
wages) lives behind a separate CSV service that needs no API key and counts
against no quota:
https://data.bls.gov/cew/data/api/{year}/{qtr}/{slice}/{code}.csv
This module is a thin wrapper over that service. It is deliberately separate
from BLSClient (different API, CSV not JSON, no key). Each call returns a list
of plain dict rows (the CSV columns), so it composes with csv/pandas directly.
Quarter argument: 1-4 for a specific quarter, or "a" for annual averages.
Area FIPS examples: "US000" national, "11000" DC (state), "11001" a county,
metro codes start with "C". Industry codes are NAICS (e.g. "10" = all
industries, "722" = food services, "622" = hospitals).
"""
import csv
import io
import requests
QCEW_BASE = "https://data.bls.gov/cew/data/api"
_HEADERS = {"User-Agent": "bls-data-library (https://gitea.doesworks.net/giteaadmin/bls-data)"}
# A few common columns, for reference / convenience:
# area_fips, own_code, industry_code, agglvl_code, size_code, year, qtr,
# annual_avg_estabs, annual_avg_emplvl, total_annual_wages, annual_avg_wkly_wage,
# avg_annual_pay (quarterly slices use month1_emplvl/qtrly_estabs/total_qtrly_wages, etc.)
OWNERSHIP = {
"total": "0", "federal": "1", "state": "2", "local": "3",
"private": "5", "government": "8",
}
def _fetch_csv(url: str) -> list[dict]:
r = requests.get(url, timeout=60, headers=_HEADERS)
r.raise_for_status()
return parse_csv(r.text)
def parse_csv(text: str) -> list[dict]:
"""Parse QCEW CSV text into a list of row dicts (BLS quotes every field)."""
return list(csv.DictReader(io.StringIO(text)))
def area(area_fips: str, year: int, qtr="a") -> list[dict]:
"""Every industry × ownership for one area (county/state/metro/national)."""
return _fetch_csv(f"{QCEW_BASE}/{year}/{qtr}/area/{area_fips}.csv")
def industry(industry_code: str, year: int, qtr="a") -> list[dict]:
"""One NAICS industry across all areas."""
return _fetch_csv(f"{QCEW_BASE}/{year}/{qtr}/industry/{industry_code}.csv")
def size(size_code: str, year: int) -> list[dict]:
"""Establishment-size breakdown (size data is published for Q1 only)."""
return _fetch_csv(f"{QCEW_BASE}/{year}/1/size/{size_code}.csv")
def filter_rows(rows, own_code=None, industry_code=None, agglvl_code=None) -> list[dict]:
"""Filter QCEW rows by ownership / industry / aggregation-level code."""
out = rows
if own_code is not None:
own = OWNERSHIP.get(own_code, own_code)
out = [r for r in out if r.get("own_code") == own]
if industry_code is not None:
out = [r for r in out if r.get("industry_code") == industry_code]
if agglvl_code is not None:
out = [r for r in out if r.get("agglvl_code") == agglvl_code]
return out