YPMS/PHASE0_STATUS.md

# Phase 0 — Handoff status

**Last updated:** 2026-10-01
**Phase:** 0 — Foundations (in flight)
**Owner:** Chris Tonjes, DOES CIO (System Owner / ISSO / AO)
**Companion docs:** [`PHASE0_PLAN.md`](PHASE0_PLAN.md) (scope + milestones), [`ARCHITECTURE.md`](ARCHITECTURE.md) (design), [`README.md`](README.md) (runnable quickstart)

---

## TL;DR

Phase 0 milestones **0.1 → 0.7** are shipped. **0.8 (CI pipeline)** is next, then **0.9 (docs + acceptance)**. All ATO artifacts (SSP, control mapping, POA&M, PIA) are drafted at v0.1–v0.3 and awaiting review signatures. The repo builds, tests are green (**156 passing, 95.91% coverage**), and Docker Compose boots. `main` is pushed to Gitea.

---

## Current state

| Dimension | Status |
|---|---|
| Phase | 0 (Foundations), milestones 0.1–0.7 shipped |
| Git branch | `main`, tracking `origin/main` (Gitea `https://gitea.doesworks.net/chris.tonjes/YPMS.git`) |
| Tests | **156 passing**, **95.91% coverage** (floor 80% enforced in CI) |
| Lint / types | `ruff` clean, `mypy` clean (43 files, strict) |
| Runtime | FastAPI + Postgres on Docker Compose; boots to `/healthz` + `/readyz`; `/auth/*` and `/registrants/*` live; 26 stub entities return 501 behind RBAC + audit |
| Migrations | 4 revisions (`0001`–`0004_stub_entity_tables`); all apply cleanly to SQLite in the drift test; live Postgres run pending (no Docker on the dev workstation) |
| Seed | 8 roles + permissions matrix + bootstrap admin (`chris` / `DOES_SYSTEM_ADMIN`) |
| ATO docs | SSP v0.3, control mapping v0.2, POA&M v0.1, PIA v0.1 — all drafts, awaiting signatures |

---

## What's done in Phase 0

Read [`PHASE0_PLAN.md`](PHASE0_PLAN.md) for the full scope tables. High points by milestone:

### 0.1 — Repo scaffold + Docker Compose ✅

- `pyproject.toml` (Python 3.12 / FastAPI / SQLAlchemy 2.x / Alembic / Postgres / dev extras: pytest, ruff, mypy, bandit, pip-audit)
- Multi-stage `Dockerfile`, `docker-compose.yml` (app on :8000, Postgres on :5433 to avoid clashing with OWC on :5432, Adminer on :8081)
- `alembic.ini` + empty baseline migration
- `src/ypms/main.py` — FastAPI factory with `/healthz` + `/readyz`
- `src/ypms/settings.py`, `logging.py` (structlog + `sanitize_for_log`), `db.py`, `middleware.py` (RequestId)
- `.github/workflows/ci.yml` — lint, types, tests+80% coverage, bandit, pip-audit, gitleaks
- `README.md`, `CONTRIBUTING.md`

### 0.2 — Framework vendor-in ✅

Vendored from OWC (`../owc-system/src/owc/`, 2026-04-20 snapshot) with `owc → ypms` renames:

- `framework/{base,bulk,crud,pagination,router,version_history}.py`
- `security/{models,rbac}.py` (Permission table + `can()`/`require_permission()` + `DOES_SYSTEM_ADMIN` super-role)
- `audit/{models,service}.py` (append-only AuditLog + `record()`)
- `users/models.py` (User/Role/UserRole with YPMS additions: `mfa_verified`, `assurance_level`; dropped OWC's `claimant_id`)
- `auth/deps.py` (stub — replaced in 0.3)

YPMS-specific addition: `framework/pii_audit.py` — `emit_pii_read()` for source RFP `C.5.6.9` #5 / NIST AU-14.

**Upstream bugfix in the vendored code:** `crud.py` pagination `next_cursor` was the peek-ahead row's id, causing silent row-drop on page boundaries. Fixed here; flagged for OWC.

Migration `0002_framework_tables.py` creates: `users`, `roles`, `user_roles`, `permissions`, `audit_log`, `version_history`.

### 0.3 — Auth + JWT + `iam.assurance_level` claim ✅

- `src/ypms/auth/security.py` — `hash_password`/`verify_password` (pbkdf2_sha256), `create_access_token` (with YPMS claims), `create_refresh_token`, `decode_token`
- `src/ypms/auth/rate_limit.py` — SlidingWindow; login limiter tuned to NIST AC-7 / source RFP `C.5.7.10`: 5 failed attempts / IP / 15 min → 429
- `src/ypms/auth/schemas.py` — LoginResponse, RefreshRequest/Response, ChangePasswordRequest, UserRead
- `src/ypms/auth/deps.py` — real `get_current_user`
- `src/ypms/auth/router.py` — `POST /auth/login`, `POST /auth/refresh`, `GET /auth/me`, `POST /auth/change-password`
- Wired into `main.py`

Access token carries: `sub`, `uid`, `roles`, `iam.assurance_level`, `mfa_verified`, `iat`, `exp`, `type`.

### 0.4 — RBAC seed loaded ✅

- 8 roles defined in `src/ypms/seed/roles.py`
- Permissions matrix at `src/ypms/seed/permissions_matrix.json` — role-first grants (30 entities × 8 actions)
- Loader at `src/ypms/seed/permissions.py` — validates every cell, refuses unknown roles at seed time, idempotent
- Runner at `src/ypms/seed/run.py` — `python -m ypms.seed.run`; auto-seeds `chris` as bootstrap admin
- Docker Compose command now: `alembic upgrade head && python -m ypms.seed.run && uvicorn …`

Guardrails baked in: `OYP_STAFF` can't see FERPA/CFSA (that's `ELIGIBILITY_EXAMINER`); `PAYROLL_ADMIN` can't either; `REPORTING_ANALYST` is strictly read-only; external portal roles never touch admin surfaces.

### 0.5 — Registrant vertical slice ✅

- `src/ypms/youth/{models,schemas,router}.py` + `migrations/versions/0003_registrants.py`
- `src/ypms/security/column_crypto.py` — AES-256-GCM, HKDF-SHA256 key from `COLUMN_ENCRYPTION_KEY` (≥32 bytes enforced), per-value nonce, column name as AAD, `v1.` prefix for rotation
- SSN is **write-only** on the model (`registrant.ssn = ...` encrypts + sets `ssn_last4`); the only read path is `decrypt_ssn()`. Serializer exposes `ssn_masked` only.
- `pii_read` audit on GET, list (one row per page, ids in `after.ids`), and history — `framework/pii_audit.py` gained `emit_pii_list_read`
- Own-row scoping for portal-only callers via nullable `Registrant.user_id` (set by Phase 1 account-link flow; not API-writable). PARENT_USER sees nothing until Guardian lands.
- Hand-built router on `CRUDService` instead of `make_router` — needs PII-read audit, own-row scoping, and `VersionHistory`-gated history. No bulk endpoints for Registrant.
- **Framework changes** (flag to OWC on re-vendor): `crud._to_dict` honors `__audit_redact__` (ciphertext → `[REDACTED]` in audit + version history) and serializes `date`; `CRUDService.history` annotation uses `builtins.list` (the `list` method shadowed it for mypy).
- Evidence trace: [`docs/coverage/registrant.md`](docs/coverage/registrant.md) — also lists what was deferred (banking columns, SSN blind index, Search route, Guardian scoping).

### 0.6 — 501-stub routers ✅

- `src/ypms/framework/stub_router.py` — `make_stub_router(entity, prefix)`; one route per matrix action (`STUB_ROUTES`), `/search` registered before `/{id}`. Each route: real `require_permission` → audit row (`action` = matrix action, `after.status = "stub_invoked"`) → 501. Never reads a request body.
- `src/ypms/stubs.py` — `STUB_ENTITIES` registry (entity → path), included in `main.py`. To build an entity for real: remove it here, register its real router.
- `tests/test_stubs.py` — matrix JSON is the oracle; sweeps 26 entities × 8 roles × 8 routes. Also pins EducationRecord / FosterIndicator closed to non-examiner roles.
- Registrant restore now requires the matrix's `Restore` action (was `Edit`).
- Test infra: `roles` / `make_user` / `api` fixtures live in `tests/conftest.py`; `api` uses one app per session.

### 0.7 — Retention scaffold ✅

- `src/ypms/framework/retention.py` — `RetentionPolicy` (data class, disposal, `period_days`, anchor column, cadence, `approved`, authority) with a derived status: `retain` / `unset` / `interim` / `approved`. `run_retention()` + `python -m ypms.framework.retention`.
- **Report-only in Phase 0.** Interim-period rows are counted as `interim_eligible`; approved-period rows as `to_purge`; nothing is ever modified. Each run logs `"N records to purge"` and writes one `retention_run` audit row (actor null).
- `src/ypms/retention_classes.py` — PIA §9 classes at interim periods, **all `approved=False`** until `YPMS-POAM-054` signs. Unset today: consent (derived from related record, Phase 1), host records, operational logs, accounts (`-071`). Signing off = flip `approved=True` and cite the schedule in `authority`; `test_nothing_is_approved_before_records_officer_signoff` must be updated in the same commit.
- `src/ypms/stub_models.py` + `migrations/versions/0004_stub_entity_tables.py` — id + mixins for the 26 stub entities. Domain phases move each class into its module; the table name stays.
- `src/ypms/models.py` — the one import-all list (Alembic env, retention job, test fixtures).
- `tests/framework/test_retention.py::test_migrations_match_models` — upgrades a scratch SQLite DB to head and diffs against `Base.metadata`. Verified it fails on an unmigrated column.

---

## What's next

### Immediate next milestone: 0.8 — CI pipeline

`.github/workflows/ci.yml` already runs ruff, mypy, pytest (80% floor), bandit, pip-audit, gitleaks. 0.8 is getting it green on the actual CI host and adding SBOM (CycloneDX; `YPMS-POAM-042`). **Open question first:** the repo lives on Gitea, so GitHub Actions won't run as-is — Gitea Actions (largely workflow-compatible) vs. Azure DevOps (ARCH §9.3 says match OWC). Confirm what OWC uses before building.

### After 0.8

- **0.9** — Docs + acceptance (README expanded, `docs/nist-mapping.md` skeleton, "Phase 0 done" acceptance checklist)

Then Phase 1 (applications + identity + parental consent) starts.

### Schema follow-up (surfaced by 0.7 drift test)

Migration `0002_framework_tables` creates named unique constraints (`uq_roles_code`, `uq_users_username`, `uq_users_email`) **and** unique indexes on the same columns; the models declare only the indexes. Harmless (double enforcement on Postgres) but real drift. 0002 is immutable — options are a `0005` that drops the three constraints, or leave as-is. Allow-listed in `KNOWN_0002_DRIFT` until decided.

### Permission-matrix review items (surfaced by 0.6)

The stub sweep runs every role against every action, so these are now visible. **Not changed** — they're policy calls for the Phase 0 review:

| # | Finding | Effect today | Likely owner |
|---|---|---|---|
| 1 | No role holds `Restore` on any entity | Only DOES_SYSTEM_ADMIN can undo a soft delete | AO + OYP |
| 2 | `Program` and `EligibilityRule` Create/Edit held by no role | Program configuration is admin-only | OYP (program-config role?) |
| 3 | `IntegrationEvent` held by no role | No integration-monitoring view outside admin | DOES OIT ops |
| 4 | HOST_USER holds `Delete` on `Position` and `JobPosting` | Safe only with row-level ownership filtering (Phase 2+; gotcha 7) | Revisit when host module is built |

---

## Key decisions locked in

Documented decisions that a future session must not re-open without a written reason:

1. **Stack: Python 3.12 / FastAPI / SQLAlchemy 2.x / Postgres** (OWC alignment). Decided 2026-09-30; recorded in [`CLAUDE.md`](CLAUDE.md).
2. **Hosting: Azure Government Cloud** (aligned with OWC and Employer 360).
3. **Chris Tonjes serves as System Owner + ISSO + AO** in Phase 0. Role concentration recorded and accepted in [`SYSTEM_SECURITY_PLAN.md`](SYSTEM_SECURITY_PLAN.md) §5 with rationale (DC agency structure + DC OCTO CISO / USDOL / OIG external oversight + independent SCA and pentest).
4. **Framework vendored from OWC** (not submoduled). Divergences carry written reasons; upstream re-vendoring is a deliberate act.
5. **`DOES_SYSTEM_ADMIN` is the super-role** and does **not** carry permission rows — it bypasses via `security.rbac.ADMIN_ROLE`.
6. **Row-level RBAC is Phase 2+** — Phase 0 is action-on-entity-type only.
7. **Every state-mutating route emits an audit row**; every stub still fires RBAC + audit; sensitive PII reads emit via `emit_pii_read`.
8. **Migrations are immutable once merged** — new revisions, never edits to older ones.
9. **SSL wording in source RFP is treated as TLS 1.2+ (target 1.3)**; FIPS 140-2 is treated as FIPS 140-3 forward alignment; `C.5.7.6` "shared secret answer" is treated as MFA-required. Documented in SSP Appendix D and control-mapping §5.
10. **Data-sharing agreements (OSSE, CFSA, SSA) are renewals of existing DOES agreements**, not new drafts (YPMS-POAM-020, -021, -022).

---

## Open questions

Each has a POA&M ID. See [`POAM.md`](POAM.md) for the full list of items. These are the ones actively awaiting a decision:

| # | Question | POA&M | Owner |
|---|---|---|---|
| 1 | What is "Level 3 security" in source RFP `C.5.7.24` — internal maturity vs. NIST 800-63 AAL/IAL 3? | `YPMS-POAM-051` | AO |
| 2 | Youth identity model for minors — social-media login scope; IAL2 proofing before payroll; parental co-auth for under-18? | `YPMS-POAM-050`, `YPMS-POAM-033` | AO + Privacy Officer + OYP |
| 3 | PCI DSS scope — expected out of scope since payroll is ACH via vendor; confirm at vendor selection | `YPMS-POAM-052` | OYP + procurement |
| 4 | Dormant COVID-era health-record CSV path — in-boundary steady state, or defer to emergency-scoped extension? | `YPMS-POAM-053` | AO + Privacy Officer |
| 5 | Retention policy for audit logs, case records, backups — align with DOES records schedule | `YPMS-POAM-054` | DOES records officer |
| 6 | SORN applicability determination | `YPMS-POAM-094` | Privacy Officer + OGC |

---

## Repo state

### Git

- Branch: `main`
- Remote: `origin` → `https://gitea.doesworks.net/chris.tonjes/YPMS.git` (HTTPS + Gitea access token)
- `main` tracks `origin/main`. Push with `git -C /Users/christonjes/cc/ypms push`.
- Milestone history: `git log --oneline` — one "Phase 0 milestone 0.N" commit per milestone, followed by a "Mark milestone 0.N complete" docs commit.

### Local git identity (repo-local, not global)

```
Chris Tonjes <ctonjes@gmail.com>
```

If commits should attribute to a DOES email instead, run:
```bash
git -C /Users/christonjes/cc/ypms config user.email <addr>
```

### Test / coverage

| Component | Coverage |
|---|---|
| `framework/` (base, bulk, crud, pagination, pii_audit, version_history) | 91–100% |
| `security/` (models, rbac) | 80–100% |
| `auth/` (security, deps, router, schemas, rate_limit) | 80–100% |
| `seed/` (roles, permissions, run) | 77–100% |
| `youth/` (models, router, schemas) | 99–100% |
| `security/column_crypto.py` | 100% |
| `framework/stub_router.py`, `stubs.py` | 100% |
| `framework/retention.py`, `retention_classes.py`, `stub_models.py` | 99–100% |
| Overall | **95.91%** |

`framework/router.py` is still excluded from coverage — Registrant uses a hand-built router, so `make_router` gets its first real callers in 0.6. Coverage config in `pyproject.toml`.

---

## How to run locally

### Docker (canonical)

```bash
cd /Users/christonjes/cc/ypms
cp .env.example .env
docker compose up --build
```

- App: http://localhost:8000 (OpenAPI at `/docs`)
- Adminer: http://localhost:8081 (server `db`, user/pw/db all `ypms`)
- On first boot, seeding runs and `chris` is created with password `change-me-immediately` (or `SEED_ADMIN_PASSWORD` env). Change it via `POST /auth/change-password`.

### Native (no Docker)

```bash
cd /Users/christonjes/cc/ypms
python3 -m venv .venv && source .venv/bin/activate
pip install -e '.[dev]'
pytest
```

Tests use SQLite in-memory — no Postgres required.

### Log in from the CLI

```bash
curl -X POST http://localhost:8000/auth/login \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d 'username=chris&password=change-me-immediately'
```

Response includes `access_token`. Use it as `Authorization: Bearer <token>`.

---

## How to verify a milestone before committing

```bash
source .venv/bin/activate
ruff check src tests           # style
mypy src                       # types (strict)
pytest -q --cov=src/ypms --cov-report=term --cov-fail-under=80
```

The GitHub Actions workflow at `.github/workflows/ci.yml` runs the same commands plus `bandit`, `pip-audit`, and `gitleaks`.

---

## Gotchas — things that would trip up someone new

1. **`app = create_app()` at the bottom of `src/ypms/main.py` runs at import time.** Tests must have env vars set before that module is imported. `tests/conftest.py` does this at *module* level (not in a fixture) — do not move it back into a fixture.

2. **SQLite in-memory tests need `StaticPool` + `check_same_thread=False`.** `TestClient` runs FastAPI handlers in a worker thread. The `db` fixture in `tests/conftest.py` configures the engine correctly — don't strip those options.

3. **`framework/router.py` deliberately does NOT use `from __future__ import annotations`.** FastAPI introspects route parameter types via `typing.get_type_hints`, which would fail to resolve closure-local TypeVars under PEP 563. See the module docstring.

4. **Two mypy overrides in `pyproject.toml`** — `framework/router.py` (FastAPI closure typevars) and `framework/crud.py` (generic-CRUD TypeVar attribute access). Same limitations OWC accepts.

5. **The vendored OWC framework had a pagination bug** — `next_cursor` was the peek-ahead row's id, silently dropping one row per page boundary. **Fixed in YPMS's copy** at `framework/crud.py`. If OWC is re-vendored, apply the fix again.

6. **`DOES_SYSTEM_ADMIN` does NOT appear in `permissions_matrix.json` under `grants`.** It bypasses via `security.rbac.ADMIN_ROLE`. Adding rows for it would be redundant and confuse the "who really has what" story.

7. **Row-level scoping is a Phase 2+ concern.** `HOST_USER` has `Host.Edit` in the matrix, which means the RBAC check passes. The service layer must ALSO filter by ownership before applying the edit. Don't rely on the matrix alone for row-level authorization.

8. **Alembic baseline (0001) is empty by design.** All schema lives in 0002+ so that "the framework tables" can be re-derived as an additive layer during future architecture reviews.

9. **`Registrant.ssn` has no getter.** Reading it raises `AttributeError` on purpose. Use `decrypt_ssn()` for the one sanctioned plaintext path, and never put it in a response schema or log line.

10. **Any new encrypted column must be listed in the model's `__audit_redact__`** — otherwise its ciphertext lands in `audit_log.before/after` and `version_history.snapshot`.

11. **`COLUMN_ENCRYPTION_KEY` changes make existing ciphertext unreadable.** Phase 0 has no rotation; don't change the dev key on a database you want to keep. `column_crypto._aead` is `lru_cache`d — tests that change the key must `cache_clear()` it and `get_settings`.

12. **The `api` test fixture shares one app across the session.** Tests must not mutate app state other than `dependency_overrides[get_db]` (which the fixture resets). If a test needs custom routes or settings, build its own app with `create_app()`.

13. **Adding an entity to `permissions_matrix.json` fails `test_every_matrix_entity_is_real_or_stubbed`** until it's either stubbed in `stubs.py` or has a real router (then add it to `REAL_ENTITIES` in that test).

14. **Every mapped table must resolve a retention policy** — including test-only models (`tests/framework/_fixtures.py::Thing` declares one). A new model without `retention_policy` fails `test_every_mapped_table_has_a_policy` and makes the job raise `LookupError`.

15. **New model modules go in `src/ypms/models.py`.** Alembic, the retention job, and the test DB all read that list.

---

## File map

Read in this order if you're picking this up cold:

1. [`CLAUDE.md`](CLAUDE.md) — project router, distinctions, open questions, resolved decisions
2. [`OVERVIEW.md`](OVERVIEW.md) — programs served, users, integrations, timeline
3. [`ARCHITECTURE.md`](ARCHITECTURE.md) — design blueprint (§4 rationale, §5 layers, §6 data, §12 delivery phases)
4. [`PHASE0_PLAN.md`](PHASE0_PLAN.md) — phase scope, milestone exit criteria, controls-wired-in-Phase-0
5. **This file** — where we are + what's next + gotchas
6. [`README.md`](README.md) — runnable quickstart
7. [`CONTRIBUTING.md`](CONTRIBUTING.md) — hard rules

Then the ATO artifacts as needed:

- [`SYSTEM_SECURITY_PLAN.md`](SYSTEM_SECURITY_PLAN.md) v0.3 — SSP
- [`NIST_800_53_CONTROL_MAPPING.md`](NIST_800_53_CONTROL_MAPPING.md) v0.2 — control mapping
- [`PRIVACY_IMPACT_ASSESSMENT.md`](PRIVACY_IMPACT_ASSESSMENT.md) v0.1 — PIA
- [`POAM.md`](POAM.md) v0.1 — 62 items, 2 closed, 4 partially closed
- [`OLA_RESPONSE_2026-09-30.md`](OLA_RESPONSE_2026-09-30.md) — external Q&A snapshot

---

## POA&M state (as of 2026-09-30)

- **62 items total** — 2 fully closed, 4 partial closures under `YPMS-POAM-040`
- **6 open decisions** (see Open Questions table above)
- **Immediate remaining action items:**
  - PIA route to Privacy Officer + OGC review
  - Pull OSSE / CFSA / SSA existing agreements from OGC files to scope renewals
  - Draft FIPS 199 categorization memo (`YPMS-POAM-001`) for AO signature
  - Schedule DC OCTO security architecture review

---

## Signals to watch for as we approach milestone 0.8

- Don't weaken a gate to get green (no blanket bandit skips, no `--cov-fail-under` drop). Suppressions are per-finding with a reason.
- The drift test runs Alembic inside pytest; CI needs the repo root as cwd so `alembic.ini` resolves.
- Retention execution (actual disposal) is Phase 5 work, alongside the OSSE/CFSA ingest pipelines. Don't add it in passing.
- Any entity that will hold PII in Phase 1+ (Guardian, TaxDocument, Attachment) should get the Registrant treatment then — `pii_read` on reads, `__audit_redact__` on encrypted columns.

---

**End of handoff v0.4.**