Add QCEW CSV module, ECEC breakdown, response caching + typed errors
Item 1 — QCEW county/industry detail: - new bls_client/qcew.py: thin client over the QCEW Open Data CSV service (area/industry/size slices; no API key, no quota), with filter_rows helper - covers the county/industry detail the timeseries API can't reach Item 2 — ECEC benefit breakdown: - new series.ecec() builder from the authoritative cm.estimate decode table - real helpers: wages, total benefits, paid leave, supplemental pay, health insurance, retirement & savings, legally required + ecec_dashboard() - FIX: ecec_total_benefits was CMU1036... (education/health industries only); correct all-civilian total benefits is CMU1030000000000D Item 3 — caching + typed errors: - bls_client/cache.py FileCache (on-disk, TTL); BLSClient(cache=True), queries_used counter (cache hits don't spend quota) - bls_client/errors.py: BLSError / BLSQuotaError / BLSRequestError; quota exhaustion now raises a clear BLSQuotaError instead of a generic RuntimeError Tests: 119 offline (+24) incl. ECEC goldens, QCEW fixture parse, cache/quota behavior; live smoke extended to ECEC + QCEW. Helper sweep: 96/96 live. README: caching/QCEW usage, updated coverage + limitations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
53
README.md
53
README.md
@ -64,12 +64,15 @@ pytest -m live # also hit the live API (needs network +
|
||||
|
||||
```
|
||||
bls_client/
|
||||
├── client.py BLSClient — batching, named fetches, catalog metadata, row flattening
|
||||
├── series.py low-level series-ID builders (LAUS, CES, CPI, PPI, OES, JOLTS, ECI, QCEW, productivity)
|
||||
├── client.py BLSClient — batching, caching, named fetches, catalog metadata, row flattening
|
||||
├── series.py low-level series-ID builders (LAUS, CES, CPI, PPI, OES, JOLTS, ECI, ECEC, QCEW, productivity)
|
||||
├── qcew.py QCEW Open Data CSV client (county/industry detail; no key, no quota)
|
||||
├── cache.py on-disk response cache (FileCache)
|
||||
├── errors.py BLSError / BLSQuotaError / BLSRequestError
|
||||
└── queries/
|
||||
├── employment.py payrolls, unemployment (LAUS), JOLTS, QCEW
|
||||
├── employment.py payrolls, unemployment (LAUS), JOLTS, QCEW totals
|
||||
├── prices.py CPI, PPI, average prices, import/export prices
|
||||
├── wages.py OES occupational wages, ECI, ECEC
|
||||
├── wages.py OES occupational wages, ECI, ECEC breakdown
|
||||
└── productivity.py major-sector productivity & costs
|
||||
```
|
||||
|
||||
@ -96,20 +99,40 @@ for the series-ID decode tables.
|
||||
| PPI — producer prices | all commodities, final demand, food, energy | ✅ |
|
||||
| OES — occupational wages | employment + wage percentiles by SOC; 15 common occupations | ✅ |
|
||||
| ECI — employment cost index | total comp / wages / benefits × civilian/private/gov | ✅ |
|
||||
| ECEC — employer cost levels | total compensation, total benefits | ✅ (totals only) |
|
||||
| ECEC — employer cost levels | full breakdown: comp, wages, benefits, paid leave, supplemental, health insurance, retirement, legally-required | ✅ |
|
||||
| Productivity & costs | output/hr, ULC, comp, hours × business/nonfarm/manufacturing | ✅ |
|
||||
| QCEW — quarterly census | national & state private totals | ⚠️ totals only via this API |
|
||||
| QCEW — quarterly census | national/state totals (timeseries) **plus** full county/industry detail via the CSV module (`bls_client.qcew`) | ✅ |
|
||||
|
||||
## Known limitations / what "complete" would add
|
||||
## Caching & quota handling
|
||||
|
||||
- **QCEW** is only partially served by the BLS *timeseries* API used here (national and
|
||||
state-level totals work). County- and industry-level QCEW detail requires the separate
|
||||
**QCEW Open Data API** (CSV: `https://data.bls.gov/cew/data/api/...`). Not yet wired up.
|
||||
- **ECEC benefit subcomponents** (health insurance, retirement & savings, etc.) need
|
||||
specific benefit-subcell codes from the ECEC component list; only the compensation and
|
||||
benefits totals are currently exposed.
|
||||
- **No caching / rate-limit handling.** Repeated runs spend against the 500/day quota; a
|
||||
small on-disk cache and a friendly error on `REQUEST_NOT_PROCESSED` (quota hit) would help.
|
||||
```python
|
||||
client = BLSClient(API_KEY, cache=True) # on-disk cache at ~/.cache/bls (1-day TTL)
|
||||
client.fetch(...) # repeated identical requests don't re-spend quota
|
||||
client.queries_used # network calls made this session (cache hits excluded)
|
||||
```
|
||||
|
||||
A blown daily quota raises `BLSQuotaError` (vs `BLSRequestError` for a bad request), so
|
||||
"am I throttled or is my series ID wrong?" is no longer ambiguous.
|
||||
|
||||
## QCEW county/industry detail
|
||||
|
||||
The timeseries helpers give QCEW national/state totals; the `qcew` module reaches the full
|
||||
county- and industry-level detail via BLS's separate CSV service (no key, no quota):
|
||||
|
||||
```python
|
||||
from bls_client import qcew
|
||||
rows = qcew.area("11000", 2024, "a") # everything for DC, annual
|
||||
dc_private = qcew.filter_rows(rows, own_code="private", industry_code="10", agglvl_code="51")
|
||||
hospitals = qcew.industry("622", 2024, "a") # one NAICS across all areas
|
||||
```
|
||||
|
||||
## Known limitations
|
||||
|
||||
- **No automatic retry/backoff** on transient network errors (a `requests` failure surfaces
|
||||
directly). Caching mitigates repeat load but there's no rate-limit pacing.
|
||||
- **Coverage is the headline cut** of each survey, not an exhaustive mirror — e.g. CES is
|
||||
national supersectors + a couple of states; CPI is the common items; OES ships 15 named
|
||||
occupations (any SOC works via `occupation_*`). Broaden the helper dicts as needed.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user