DOES-Applications/bls-data
5
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5c8e0f11f3ed9005a99b3422f77e3404e958b5f7
bls-data/USAGE.md
Dave Boyd 4fe734381e Fix broken series-ID builders; env-var config; project README 
Repair every dead series-ID encoding (library now 82/82 query helpers
return live data, verified against the BLS API):

- JOLTS: 21-char format (was 18) — add state/area/sizeclass fields
- OES: national area code 0000000 (was invalid 0000400)
- ECI: correct owner/component/estimate encoding, default unadjusted (CIU)
- Productivity: 4-digit sector + 4-digit measure codes (was 2+3)
- QCEW: 13-char timeseries-API form (ENUUS00010510 / ENU{fips}00010{own}10)
- PPI: repoint finished-goods -> final demand (WPUFD4); keep alias
- ECEC: drop fabricated health-insurance/retirement helpers; add total benefits
- wages SOC: software developers 151132 -> 151252 (2018 SOC)

Tooling/docs:
- config reads BLS_API_KEY env var (takes precedence; config.py gitignored)
- add requirements.txt
- rewrite README as project front door + coverage table + limitations
- correct JOLTS/OES tables in series_id_formats.md, USAGE.md, dataset explorer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-22 09:45:27 -04:00

369 lines
12 KiB
Markdown
Raw Blame History

# BLS Data Library — Usage Guide
A Python library for querying the BLS Public Data API v2, with pre-built series for employment, prices, wages, and productivity.
---
## Data Flow
```mermaid
flowchart TD
subgraph YOUR_CODE ["Your Code"]
direction TB
Q["Query functions\nbls_client/queries/\nemployment · prices · wages · productivity"]
S["Series ID builders\nbls_client/series.py\nlaus_state() · ces_national() · cpi() · jolts()"]
C["BLSClient\nbls_client/client.py\nfetch() · fetch_named() · to_rows()"]
Q -->|"returns series ID string"| C
S -->|"returns series ID string"| C
end
subgraph BLS_API ["BLS Public Data API — api.bls.gov/publicAPI/v2"]
direction TB
EP1["/timeseries/data/\nPOST · up to 50 series · 20 years"]
EP2["/surveys\nGET · all 68 survey codes"]
EP3["/timeseries/popular?survey=XX\nGET · top series per survey"]
end
subgraph OUTPUT ["Output"]
R["dict keyed by series ID\nor labeled dict via fetch_named()"]
ROW["Flat row list\nclient.to_rows() → CSV / DataFrame"]
HTML["HTML Reports\ngenerate_report.py\nbls_dataset_explorer.py"]
end
C -->|"POST JSON\nwith API key + series IDs"| EP1
C -->|GET| EP2
C -->|GET| EP3
EP1 -->|"JSON: value · period · footnotes\n± catalog · calculations"| R
R -->|"client.to_rows()"| ROW
R -->|"passed to report generator"| HTML
```
---
## API Limits
| | Unregistered (v1) | Registered (v2) |
|---|---|---|
| Daily queries | 25 | **500** |
| Series per call | 25 | **50** |
| Years per call | 10 | **20** |
| Catalog metadata | No | Yes |
| MoM/YoY calculations | No | Yes |
Register free at **https://data.bls.gov/registrationEngine/**
---
## Setup
```bash
# 1. Copy and fill in your API key
cp config.example.py config.py
# Edit config.py and paste your key
# 2. Install dependencies (standard library only + requests)
pip install requests
```
---
## Quick Start
```python
from config import BLS_API_KEY
from bls_client import BLSClient
from bls_client.queries import employment, prices
client = BLSClient(BLS_API_KEY)
# Fetch a single series
result = client.fetch_latest(employment.nonfarm_payrolls(), years=2)
# Fetch multiple named series at once
results = client.fetch_named(prices.cpi_dashboard(), 2023, 2025)
for label, s in results.items():
obs = client.latest_obs(s)
print(f"{label}: {obs['value']} ({obs['periodName']} {obs['year']})")
```
---
## BLSClient API
### `fetch(series_ids, start_year, end_year, **kwargs)`
Fetch one or more series. Handles batching automatically (50 per API call).
```python
result = client.fetch(
["CES0000000001", "LAUST110000000000003"],
start_year=2020,
end_year=2025,
catalog=True, # include series title and metadata
calculations=True, # include MoM/YoY net and % changes
annual_average=False, # include M13 annual average row
)
# Returns: {"CES0000000001": {"data": [...], "catalog": {...}}, ...}
```
**Observation format:**
```python
{
"year": "2025",
"period": "M04", # M01-M12=monthly, Q01-Q04=quarterly, A01=annual
"periodName": "April",
"value": "6.4", # "-" when not available
"footnotes": [{"code": "R", "text": "Revised..."}],
"calculations": { # only when calculations=True
"net_changes": {"1": "-0.2", "3": "0.1", "12": "1.3"},
"pct_changes": {"1": "-3.0", "3": "1.5", "12": "25.5"},
}
}
```
**Footnote codes:**
| Code | Meaning |
|------|---------|
| R | Revised |
| P | Preliminary |
| X | Not available (government shutdown gap, etc.) |
| N | Not available |
### `fetch_latest(series_ids, years=2, **kwargs)`
Convenience wrapper — fetches the most recent N years.
```python
result = client.fetch_latest(employment.nonfarm_payrolls(), years=3)
```
### `fetch_named(named_dict, start_year, end_year, **kwargs)`
Fetch a `{label: series_id}` dict and return results keyed by label.
```python
results = client.fetch_named(
{"DC Unemployment": "LAUST110000000000003",
"MD Unemployment": "LAUST240000000000003"},
2023, 2025
)
```
### `client.to_rows(results)`
Flatten fetch results into a list of dicts for CSV/DataFrame use.
```python
rows = client.to_rows(result)
# [{"series_id": "CES0000000001", "series_title": "...",
# "year": "2025", "period": "M04", "period_name": "April", "value": "158987"}, ...]
import csv
with open("output.csv", "w", newline="") as f:
w = csv.DictWriter(f, fieldnames=rows[0].keys())
w.writeheader(); w.writerows(rows)
```
---
## Query Library
### Employment (`bls_client/queries/employment.py`)
```mermaid
graph LR
E["employment module"]
E --> NP["nonfarm_payrolls()"]
E --> PP["private_payrolls()"]
E --> GE["government_employment()"]
E --> NUR["national_unemployment_rate()"]
E --> LFP["national_labor_force_participation()"]
E --> DC["dc_unemployment_rate()"]
E --> MD["md_unemployment_rate()"]
E --> VA["va_unemployment_rate()"]
E --> SU["state_unemployment(state_fips)\nreturns dict of 4 measures"]
E --> DRU["dc_region_unemployment()\n6 geos as labeled dict"]
E --> JD["jolts_dashboard()\n5 JOLTS measures"]
E --> QCEW["qcew_national_private()\nqcew_dc_private()\nqcew_state(fips)"]
```
| Function | Series | Description |
|---|---|---|
| `nonfarm_payrolls()` | CES0000000001 | Total nonfarm payroll employment |
| `private_payrolls()` | CES0500000001 | Total private employment |
| `government_employment()` | CES9000000001 | Federal + state + local |
| `national_unemployment_rate()` | LNS14000000 | CPS U-3 rate (SA) |
| `dc_unemployment_rate()` | LAUST110000000000003 | DC, not SA |
| `state_unemployment(fips)` | 4 LAUS series | Rate/employed/unemployed/LF |
| `dc_region_unemployment()` | 6 series | DC/MD/VA + 3 MSAs |
| `job_openings_level()` | JTS000000000000000JOL | JOLTS openings |
| `jolts_dashboard()` | 5 series | Full JOLTS flow |
| `qcew_state(fips)` | EN… | QCEW quarterly by state |
### Prices (`bls_client/queries/prices.py`)
| Function | Series | Description |
|---|---|---|
| `cpi_all_items()` | CUUR0000SA0 | CPI-U headline |
| `cpi_core()` | CUSR0000SA0L1E | All items less food & energy (SA) |
| `cpi_food()` | CUUR0000SAF | Food at home + away |
| `cpi_energy()` | CUUR0000SA0E | Energy component |
| `cpi_shelter()` | CUUR0000SAH1 | Shelter/housing |
| `cpi_gasoline()` | CUSR0000SS47014 | Gasoline (SA) |
| `cpi_dashboard()` | 7 series | Full CPI labeled dict |
| `cpi_w_all_items()` | CWUR0000SA0 | CPI-W (COLA basis) |
| `ppi_all_commodities()` | WPU00000000 | PPI all commodities |
| `ppi_dashboard()` | 4 series | PPI components |
| `avg_price_electricity()` | APU000072610 | ¢/KWh retail |
| `avg_price_gasoline_regular()` | APU00007471A | $/gallon regular |
| `import_price_index()` | EIUIR | Import price index |
| `export_price_index()` | EIUIQ | Export price index |
### Wages (`bls_client/queries/wages.py`)
| Function | Series | Description |
|---|---|---|
| `eci_total_compensation()` | CIU1010000000000A | ECI civilian all workers |
| `eci_wages()` | CIU1010000000000W | ECI wages only |
| `eci_benefits()` | CIU1010000000000B | ECI benefits only |
| `eci_dashboard()` | 5 series | Civilian + private + state/local |
| `ecec_total_compensation()` | CMU1010000000000D | Cost per hour worked |
| `ecec_health_insurance()` | CMU1010000000000H | Health ins. cost/hr |
| `median_weekly_earnings()` | LEU0252881600 | Median weekly earnings (SA) |
| `occupation_annual_median_wage(soc)` | OE… | OEWS wage for SOC code |
**SOC codes** available in `wages.SOC_CODES` dict.
### Productivity (`bls_client/queries/productivity.py`)
| Function | Series | Description |
|---|---|---|
| `business_output_per_hour()` | PRS85006092 | Business sector productivity |
| `business_unit_labor_cost()` | PRS85006111 | Unit labor costs |
| `nonfarm_output_per_hour()` | PRS86006092 | Nonfarm business |
| `manufacturing_output_per_hour()` | PRS88006092 | Manufacturing |
| `productivity_dashboard()` | 4 series | Labeled dict |
---
## Series ID Builders
Use `bls_client/series.py` when you need series not covered by the query library.
```mermaid
flowchart LR
S["series.py"]
S --> LA["laus_state(state_fips, measure, seasonal)\nlaus_county(state, county, measure)\nlaus_msa(state, cbsa_code, measure)"]
S --> CE["ces_national(industry, data_type, seasonal)\nces_state(state_fips, industry, data_type)"]
S --> CP["cpi(item, area, seasonal, series)"]
S --> PP["ppi_commodity(commodity_code)"]
S --> OE["oes_national(occupation, industry, data_type)"]
S --> JT["jolts(element, rate_level, industry, ownership)"]
S --> EC["eci(worker_type, occupation, industry, component)"]
S --> PR["productivity(sector, measure, seasonal)"]
```
**Examples:**
```python
from bls_client import series
# LAUS: any geography
series.laus_state(36, "rate") # New York unemployment rate
series.laus_county(24, 33, "employed") # Howard County MD employment
series.laus_msa(51, 40060, "rate") # Richmond VA MSA rate
# CES: specific industry
series.ces_national("healthcare", "avg_hourly_earn") # healthcare wages
series.ces_state(11, "government") # DC govt employment
# CPI: any component
series.cpi("shelter", seasonal=True) # SA shelter index
series.cpi("all_items", series="W") # CPI-W all items
# JOLTS: any flow
series.jolts("quits", "R") # quits rate
series.jolts("hires", "L", seasonal=False) # hires level, NSA
# OES: any occupation
series.oes_national("291141", data_type="annual_median") # RN median wage
```
---
## Examples
| File | What it shows |
|---|---|
| `examples/basic_pull.py` | Fetch single series, named dict, flatten to rows |
| `examples/custom_series.py` | Build series IDs manually; JOLTS, OES, CPI, LAUS by state |
| `dc_md_va_unemployment.py` | LAUS pull for 6 DC-region geos → CSV |
| `generate_report.py` | Full HTML report with Chart.js charts |
| `bls_dataset_explorer.py` | All 68 surveys with live data samples |
---
## LAUS State FIPS Reference
| FIPS | State | | FIPS | State |
|------|-------|-|------|-------|
| 01 | Alabama | | 30 | Montana |
| 02 | Alaska | | 31 | Nebraska |
| 04 | Arizona | | 32 | Nevada |
| 05 | Arkansas | | 33 | New Hampshire |
| 06 | California | | 34 | New Jersey |
| 08 | Colorado | | 35 | New Mexico |
| 09 | Connecticut | | 36 | New York |
| 10 | Delaware | | 37 | North Carolina |
| **11** | **District of Columbia** | | 38 | North Dakota |
| 12 | Florida | | 39 | Ohio |
| 13 | Georgia | | 40 | Oklahoma |
| 15 | Hawaii | | 41 | Oregon |
| 16 | Idaho | | 42 | Pennsylvania |
| 17 | Illinois | | 44 | Rhode Island |
| 18 | Indiana | | 45 | South Carolina |
| 19 | Iowa | | 46 | South Dakota |
| 20 | Kansas | | 47 | Tennessee |
| 21 | Kentucky | | 48 | Texas |
| 22 | Louisiana | | 49 | Utah |
| 23 | Maine | | 50 | Vermont |
| **24** | **Maryland** | | **51** | **Virginia** |
| 25 | Massachusetts | | 53 | Washington |
| 26 | Michigan | | 54 | West Virginia |
| 27 | Minnesota | | 55 | Wisconsin |
| 28 | Mississippi | | 56 | Wyoming |
| 29 | Missouri | | | |
---
## Files in This Repository
```
bls-data/
├── USAGE.md ← this file
├── README.md ← API reference (endpoints, limits, formats)
├── config.example.py ← copy to config.py and add your key
│
├── bls_client/ ← Python library
│ ├── __init__.py
│ ├── client.py ← BLSClient class
│ ├── series.py ← series ID builder functions
│ └── queries/
│ ├── employment.py ← LAUS, CES, JOLTS, QCEW series
│ ├── prices.py ← CPI, PPI, Import/Export series
│ ├── wages.py ← OES, ECI, ECEC, CPS earnings
│ └── productivity.py ← Major sector productivity
│
├── examples/
│ ├── basic_pull.py ← Getting started
│ └── custom_series.py ← Building series IDs manually
│
├── generate_report.py ← DC/MD/VA HTML report generator
├── bls_dataset_explorer.py ← All-surveys HTML reference
├── dc_md_va_unemployment.py ← CLI pull → CSV
│
├── series_id_formats.md ← Series ID decode tables (all surveys)
├── qcew_field_schema.md ← QCEW CSV field layouts
├── surveys.json ← All 68 surveys from API
└── api_response_example.json ← Sample API response
```
Reference in New Issue View Git Blame Copy Permalink