ci / test (push) Has been cancelled
- README rewritten: Docker and native quickstarts, try-it flow, gate commands, promote-a-stub steps. - CONTRIBUTING rules 9-13 (pii_read on PII reads, __audit_redact__ for encrypted columns, retention policy per table, models.py registry, vendored-code change discipline); PR checklist updated. - docs/nist-mapping.md: per-control file:line evidence and remaining gap. - NIST_800_53_CONTROL_MAPPING.md v0.3: Phase 0 notes on 28 rows (12 implemented, 12 partial, 4 progress notes on hybrid/tailored rows) plus a status-shift table. - PHASE0_PLAN: 0.9 complete; Definition of Done 12/14 checked with how each was verified. Open: first green Gitea CI run, Docker/Postgres boot. - POA&M and status handoff updated; status TL;DR no longer claims an unverified Docker boot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
375 lines
25 KiB
Markdown
375 lines
25 KiB
Markdown
# Phase 0 — Handoff status
|
||
|
||
**Last updated:** 2026-10-01
|
||
**Phase:** 0 — Foundations (built; exit pending two infra checks)
|
||
**Owner:** Chris Tonjes, DOES CIO (System Owner / ISSO / AO)
|
||
**Companion docs:** [`PHASE0_PLAN.md`](PHASE0_PLAN.md) (scope + milestones), [`ARCHITECTURE.md`](ARCHITECTURE.md) (design), [`README.md`](README.md) (runnable quickstart)
|
||
|
||
---
|
||
|
||
## TL;DR
|
||
|
||
All nine Phase 0 milestones are built. **Phase 0 exits on two infrastructure checks**, neither of which can run on the dev workstation: (1) the first green Gitea Actions run on `main` — needs an `act_runner` registered on `gitea.doesworks.net`; (2) `docker compose up` against real Postgres. Definition of Done (PHASE0_PLAN §8) is 12 of 14 checked; everything else was verified end to end over HTTP from a clean clone. Tests green (**167 passing, ~96% coverage**); every CI gate passes locally. Control mapping is at v0.3 with file:line evidence (12 ✅ / 12 🟡). `main` is pushed to Gitea.
|
||
|
||
---
|
||
|
||
## Current state
|
||
|
||
| Dimension | Status |
|
||
|---|---|
|
||
| Phase | 0 (Foundations), milestones 0.1–0.9 built; exit pending CI runner + Docker/Postgres boot |
|
||
| Git branch | `main`, tracking `origin/main` (Gitea `https://gitea.doesworks.net/chris.tonjes/YPMS.git`) |
|
||
| Tests | **167 passing**, **~96% coverage** (floor 80% enforced in CI) |
|
||
| CI | `.gitea/workflows/ci.yml` — all gates pass locally; runs queue as "Waiting" until a runner is registered |
|
||
| Lint / types | `ruff` clean, `mypy` clean (43 files, strict) |
|
||
| Runtime | Verified booting natively (uvicorn + SQLite) from a clean clone; Docker Compose + Postgres not yet run; `/auth/*` and `/registrants/*` live; 26 stub entities return 501 behind RBAC + audit |
|
||
| Migrations | 4 revisions (`0001`–`0004_stub_entity_tables`); all apply cleanly to SQLite in the drift test; live Postgres run pending (no Docker on the dev workstation) |
|
||
| Seed | 8 roles + permissions matrix + bootstrap admin (`chris` / `DOES_SYSTEM_ADMIN`) |
|
||
| ATO docs | SSP v0.3, control mapping **v0.3 (Phase 0 evidence)**, POA&M v0.1, PIA v0.1 — all drafts, awaiting signatures |
|
||
|
||
---
|
||
|
||
## What's done in Phase 0
|
||
|
||
Read [`PHASE0_PLAN.md`](PHASE0_PLAN.md) for the full scope tables. High points by milestone:
|
||
|
||
### 0.1 — Repo scaffold + Docker Compose ✅
|
||
|
||
- `pyproject.toml` (Python 3.12 / FastAPI / SQLAlchemy 2.x / Alembic / Postgres / dev extras: pytest, ruff, mypy, bandit, pip-audit)
|
||
- Multi-stage `Dockerfile`, `docker-compose.yml` (app on :8000, Postgres on :5433 to avoid clashing with OWC on :5432, Adminer on :8081)
|
||
- `alembic.ini` + empty baseline migration
|
||
- `src/ypms/main.py` — FastAPI factory with `/healthz` + `/readyz`
|
||
- `src/ypms/settings.py`, `logging.py` (structlog + `sanitize_for_log`), `db.py`, `middleware.py` (RequestId)
|
||
- `.github/workflows/ci.yml` — lint, types, tests+80% coverage, bandit, pip-audit, gitleaks (moved to `.gitea/workflows/` in 0.8)
|
||
- `README.md`, `CONTRIBUTING.md`
|
||
|
||
### 0.2 — Framework vendor-in ✅
|
||
|
||
Vendored from OWC (`../owc-system/src/owc/`, 2026-04-20 snapshot) with `owc → ypms` renames:
|
||
|
||
- `framework/{base,bulk,crud,pagination,router,version_history}.py`
|
||
- `security/{models,rbac}.py` (Permission table + `can()`/`require_permission()` + `DOES_SYSTEM_ADMIN` super-role)
|
||
- `audit/{models,service}.py` (append-only AuditLog + `record()`)
|
||
- `users/models.py` (User/Role/UserRole with YPMS additions: `mfa_verified`, `assurance_level`; dropped OWC's `claimant_id`)
|
||
- `auth/deps.py` (stub — replaced in 0.3)
|
||
|
||
YPMS-specific addition: `framework/pii_audit.py` — `emit_pii_read()` for source RFP `C.5.6.9` #5 / NIST AU-14.
|
||
|
||
**Upstream bugfix in the vendored code:** `crud.py` pagination `next_cursor` was the peek-ahead row's id, causing silent row-drop on page boundaries. Fixed here; flagged for OWC.
|
||
|
||
Migration `0002_framework_tables.py` creates: `users`, `roles`, `user_roles`, `permissions`, `audit_log`, `version_history`.
|
||
|
||
### 0.3 — Auth + JWT + `iam.assurance_level` claim ✅
|
||
|
||
- `src/ypms/auth/security.py` — `hash_password`/`verify_password` (pbkdf2_sha256), `create_access_token` (with YPMS claims), `create_refresh_token`, `decode_token`
|
||
- `src/ypms/auth/rate_limit.py` — SlidingWindow; login limiter tuned to NIST AC-7 / source RFP `C.5.7.10`: 5 failed attempts / IP / 15 min → 429
|
||
- `src/ypms/auth/schemas.py` — LoginResponse, RefreshRequest/Response, ChangePasswordRequest, UserRead
|
||
- `src/ypms/auth/deps.py` — real `get_current_user`
|
||
- `src/ypms/auth/router.py` — `POST /auth/login`, `POST /auth/refresh`, `GET /auth/me`, `POST /auth/change-password`
|
||
- Wired into `main.py`
|
||
|
||
Access token carries: `sub`, `uid`, `roles`, `iam.assurance_level`, `mfa_verified`, `iat`, `exp`, `type`.
|
||
|
||
### 0.4 — RBAC seed loaded ✅
|
||
|
||
- 8 roles defined in `src/ypms/seed/roles.py`
|
||
- Permissions matrix at `src/ypms/seed/permissions_matrix.json` — role-first grants (30 entities × 8 actions)
|
||
- Loader at `src/ypms/seed/permissions.py` — validates every cell, refuses unknown roles at seed time, idempotent
|
||
- Runner at `src/ypms/seed/run.py` — `python -m ypms.seed.run`; auto-seeds `chris` as bootstrap admin
|
||
- Docker Compose command now: `alembic upgrade head && python -m ypms.seed.run && uvicorn …`
|
||
|
||
Guardrails baked in: `OYP_STAFF` can't see FERPA/CFSA (that's `ELIGIBILITY_EXAMINER`); `PAYROLL_ADMIN` can't either; `REPORTING_ANALYST` is strictly read-only; external portal roles never touch admin surfaces.
|
||
|
||
### 0.5 — Registrant vertical slice ✅
|
||
|
||
- `src/ypms/youth/{models,schemas,router}.py` + `migrations/versions/0003_registrants.py`
|
||
- `src/ypms/security/column_crypto.py` — AES-256-GCM, HKDF-SHA256 key from `COLUMN_ENCRYPTION_KEY` (≥32 bytes enforced), per-value nonce, column name as AAD, `v1.` prefix for rotation
|
||
- SSN is **write-only** on the model (`registrant.ssn = ...` encrypts + sets `ssn_last4`); the only read path is `decrypt_ssn()`. Serializer exposes `ssn_masked` only.
|
||
- `pii_read` audit on GET, list (one row per page, ids in `after.ids`), and history — `framework/pii_audit.py` gained `emit_pii_list_read`
|
||
- Own-row scoping for portal-only callers via nullable `Registrant.user_id` (set by Phase 1 account-link flow; not API-writable). PARENT_USER sees nothing until Guardian lands.
|
||
- Hand-built router on `CRUDService` instead of `make_router` — needs PII-read audit, own-row scoping, and `VersionHistory`-gated history. No bulk endpoints for Registrant.
|
||
- **Framework changes** (flag to OWC on re-vendor): `crud._to_dict` honors `__audit_redact__` (ciphertext → `[REDACTED]` in audit + version history) and serializes `date`; `CRUDService.history` annotation uses `builtins.list` (the `list` method shadowed it for mypy).
|
||
- Evidence trace: [`docs/coverage/registrant.md`](docs/coverage/registrant.md) — also lists what was deferred (banking columns, SSN blind index, Search route, Guardian scoping).
|
||
|
||
### 0.6 — 501-stub routers ✅
|
||
|
||
- `src/ypms/framework/stub_router.py` — `make_stub_router(entity, prefix)`; one route per matrix action (`STUB_ROUTES`), `/search` registered before `/{id}`. Each route: real `require_permission` → audit row (`action` = matrix action, `after.status = "stub_invoked"`) → 501. Never reads a request body.
|
||
- `src/ypms/stubs.py` — `STUB_ENTITIES` registry (entity → path), included in `main.py`. To build an entity for real: remove it here, register its real router.
|
||
- `tests/test_stubs.py` — matrix JSON is the oracle; sweeps 26 entities × 8 roles × 8 routes. Also pins EducationRecord / FosterIndicator closed to non-examiner roles.
|
||
- Registrant restore now requires the matrix's `Restore` action (was `Edit`).
|
||
- Test infra: `roles` / `make_user` / `api` fixtures live in `tests/conftest.py`; `api` uses one app per session.
|
||
|
||
### 0.7 — Retention scaffold ✅
|
||
|
||
- `src/ypms/framework/retention.py` — `RetentionPolicy` (data class, disposal, `period_days`, anchor column, cadence, `approved`, authority) with a derived status: `retain` / `unset` / `interim` / `approved`. `run_retention()` + `python -m ypms.framework.retention`.
|
||
- **Report-only in Phase 0.** Interim-period rows are counted as `interim_eligible`; approved-period rows as `to_purge`; nothing is ever modified. Each run logs `"N records to purge"` and writes one `retention_run` audit row (actor null).
|
||
- `src/ypms/retention_classes.py` — PIA §9 classes at interim periods, **all `approved=False`** until `YPMS-POAM-054` signs. Unset today: consent (derived from related record, Phase 1), host records, operational logs, accounts (`-071`). Signing off = flip `approved=True` and cite the schedule in `authority`; `test_nothing_is_approved_before_records_officer_signoff` must be updated in the same commit.
|
||
- `src/ypms/stub_models.py` + `migrations/versions/0004_stub_entity_tables.py` — id + mixins for the 26 stub entities. Domain phases move each class into its module; the table name stays.
|
||
- `src/ypms/models.py` — the one import-all list (Alembic env, retention job, test fixtures).
|
||
- `tests/framework/test_retention.py::test_migrations_match_models` — upgrades a scratch SQLite DB to head and diffs against `Base.metadata`. Verified it fails on an unmigrated column.
|
||
|
||
### 0.8 — CI pipeline 🟡 (green-on-main pending runner)
|
||
|
||
- `.gitea/workflows/ci.yml` (moved from `.github/`): checkout (full history) → Python 3.12 → ruff → mypy → pytest + 80% floor → bandit → `pip-audit --strict .` (runtime deps) → gitleaks 8.30.1 (pinned, sha256-verified) → CycloneDX SBOM from a clean runtime-only venv, uploaded as artifact `sbom`.
|
||
- **Running the gates for real found four things:** the old `pip-audit --disable-pip` step was a usage error and could never pass; `ecdsa` (PYSEC-2026-1325, no fix) via python-jose → swapped to **PyJWT**; PyJWT flagged an 11-byte test JWT secret → `JWT_SECRET` now **requires ≥ 32 bytes** (RFC 7518 §3.2) at settings load; bandit B101 on an `assert` in the retention job.
|
||
- Every step passes locally. Gitea has queued runs #1–#5 (and this push) as **Waiting** — no runner is online.
|
||
- **To close 0.8:** register an `act_runner` against `gitea.doesworks.net` with an `ubuntu-latest` label mapped to an act-compatible image (e.g. `catthehacker/ubuntu:act-22.04`), outbound to github.com + PyPI. Then confirm the newest run on `main` is green.
|
||
|
||
### 0.9 — Docs + acceptance ✅
|
||
|
||
- **README** rewritten: Docker and native (no-Docker) quickstarts, a try-it curl flow, all gate commands, and step-by-step "promote a stub to a real entity".
|
||
- **CONTRIBUTING** rules 9–13: PII reads emit `pii_read`; encrypted columns go in `__audit_redact__`; every table has a retention policy; model modules registered in `models.py`; vendored-code changes need a reason + upstream list entry. PR checklist and suppression rule updated.
|
||
- **`docs/nist-mapping.md`** — per-control code evidence with linked file:line and the remaining gap. **Control mapping v0.3** carries the same notes per row plus a status-shift table: 12 ✅, 12 🟡; AU-11, CM-3, SI-12 keep 🤝 and SC-8 keeps ⚠️, each with a progress note.
|
||
- **Closed gaps found while preparing evidence** (each promised by PHASE0_PLAN §6/§7 but not yet built):
|
||
- **Failed logins weren't audited** (AU-2, C.5.7.39). Now `login_failed` with reason `bad_credentials` / `unknown_user` / `inactive` / `throttled`. The typed username is stored only when it matches a real account.
|
||
- **SI-11:** app-wide handler returns `{"error":"internal_error","request_id":…}`; logs type + stack frames, never the message (driver errors echo PII); engine `hide_parameters=True`.
|
||
- **Startup retention check:** `create_app` refuses to boot if any table lacks a policy.
|
||
- **CM-3:** `migrations/IMMUTABLE.sha256` pins merged revisions; test fails on edits.
|
||
- Alembic `fileConfig(disable_existing_loggers=False)` — in-process migrations were silencing app loggers in tests.
|
||
- **End-to-end check** from a clean clone: migrate → seed → uvicorn → login, Registrant create/read/update/delete, stub 501, audit trail, ciphertext-only at rest, no SSN in the server log.
|
||
|
||
---
|
||
|
||
## What's next
|
||
|
||
### Phase 0 exit — two infrastructure checks
|
||
|
||
1. **CI green on `main`.** Register an `act_runner` on `gitea.doesworks.net` (label `ubuntu-latest` → an act-compatible image such as `catthehacker/ubuntu:act-22.04`; outbound to github.com and PyPI). The queued runs will start; confirm the newest on `main` passes. Closes 0.8 and the CI item in PLAN §8.
|
||
2. **Docker + Postgres boot.** On a Docker-capable machine: `cp .env.example .env && docker compose up --build`, then the README "Try it" flow. Closes the last PLAN §8 item and gives the first live Postgres run of migrations 0001–0004. If Postgres disagrees with SQLite anywhere, fix in a new migration.
|
||
|
||
Then: sign off Phase 0, and draft `PHASE1_PLAN.md` (applications + identity + parental consent — ARCH §12). Decisions that should be made before Phase 1 starts: the four permission-matrix review items below, the 0002 constraint follow-up, and `YPMS-POAM-050` (youth identity model), which Phase 1 is built around.
|
||
|
||
### Upstream to OWC (accumulated)
|
||
|
||
1. `crud.py` pagination cursor fix (0.2).
|
||
2. `crud._to_dict` `__audit_redact__` hook + `date` serialization (0.5) — OWC claimant SSN has the same exposure.
|
||
3. `CRUDService.history` `builtins.list` annotation (0.5).
|
||
4. python-jose → PyJWT to drop `ecdsa` (0.8).
|
||
|
||
### Schema follow-up (surfaced by 0.7 drift test)
|
||
|
||
Migration `0002_framework_tables` creates named unique constraints (`uq_roles_code`, `uq_users_username`, `uq_users_email`) **and** unique indexes on the same columns; the models declare only the indexes. Harmless (double enforcement on Postgres) but real drift. 0002 is immutable — options are a `0005` that drops the three constraints, or leave as-is. Allow-listed in `KNOWN_0002_DRIFT` until decided.
|
||
|
||
### Permission-matrix review items (surfaced by 0.6)
|
||
|
||
The stub sweep runs every role against every action, so these are now visible. **Not changed** — they're policy calls for the Phase 0 review:
|
||
|
||
| # | Finding | Effect today | Likely owner |
|
||
|---|---|---|---|
|
||
| 1 | No role holds `Restore` on any entity | Only DOES_SYSTEM_ADMIN can undo a soft delete | AO + OYP |
|
||
| 2 | `Program` and `EligibilityRule` Create/Edit held by no role | Program configuration is admin-only | OYP (program-config role?) |
|
||
| 3 | `IntegrationEvent` held by no role | No integration-monitoring view outside admin | DOES OIT ops |
|
||
| 4 | HOST_USER holds `Delete` on `Position` and `JobPosting` | Safe only with row-level ownership filtering (Phase 2+; gotcha 7) | Revisit when host module is built |
|
||
|
||
---
|
||
|
||
## Key decisions locked in
|
||
|
||
Documented decisions that a future session must not re-open without a written reason:
|
||
|
||
1. **Stack: Python 3.12 / FastAPI / SQLAlchemy 2.x / Postgres** (OWC alignment). Decided 2026-09-30; recorded in [`CLAUDE.md`](CLAUDE.md).
|
||
2. **Hosting: Azure Government Cloud** (aligned with OWC and Employer 360).
|
||
3. **Chris Tonjes serves as System Owner + ISSO + AO** in Phase 0. Role concentration recorded and accepted in [`SYSTEM_SECURITY_PLAN.md`](SYSTEM_SECURITY_PLAN.md) §5 with rationale (DC agency structure + DC OCTO CISO / USDOL / OIG external oversight + independent SCA and pentest).
|
||
4. **Framework vendored from OWC** (not submoduled). Divergences carry written reasons; upstream re-vendoring is a deliberate act.
|
||
5. **`DOES_SYSTEM_ADMIN` is the super-role** and does **not** carry permission rows — it bypasses via `security.rbac.ADMIN_ROLE`.
|
||
6. **Row-level RBAC is Phase 2+** — Phase 0 is action-on-entity-type only.
|
||
7. **Every state-mutating route emits an audit row**; every stub still fires RBAC + audit; sensitive PII reads emit via `emit_pii_read`.
|
||
8. **Migrations are immutable once merged** — new revisions, never edits to older ones.
|
||
9. **SSL wording in source RFP is treated as TLS 1.2+ (target 1.3)**; FIPS 140-2 is treated as FIPS 140-3 forward alignment; `C.5.7.6` "shared secret answer" is treated as MFA-required. Documented in SSP Appendix D and control-mapping §5.
|
||
10. **Data-sharing agreements (OSSE, CFSA, SSA) are renewals of existing DOES agreements**, not new drafts (YPMS-POAM-020, -021, -022).
|
||
|
||
---
|
||
|
||
## Open questions
|
||
|
||
Each has a POA&M ID. See [`POAM.md`](POAM.md) for the full list of items. These are the ones actively awaiting a decision:
|
||
|
||
| # | Question | POA&M | Owner |
|
||
|---|---|---|---|
|
||
| 1 | What is "Level 3 security" in source RFP `C.5.7.24` — internal maturity vs. NIST 800-63 AAL/IAL 3? | `YPMS-POAM-051` | AO |
|
||
| 2 | Youth identity model for minors — social-media login scope; IAL2 proofing before payroll; parental co-auth for under-18? | `YPMS-POAM-050`, `YPMS-POAM-033` | AO + Privacy Officer + OYP |
|
||
| 3 | PCI DSS scope — expected out of scope since payroll is ACH via vendor; confirm at vendor selection | `YPMS-POAM-052` | OYP + procurement |
|
||
| 4 | Dormant COVID-era health-record CSV path — in-boundary steady state, or defer to emergency-scoped extension? | `YPMS-POAM-053` | AO + Privacy Officer |
|
||
| 5 | Retention policy for audit logs, case records, backups — align with DOES records schedule | `YPMS-POAM-054` | DOES records officer |
|
||
| 6 | SORN applicability determination | `YPMS-POAM-094` | Privacy Officer + OGC |
|
||
|
||
---
|
||
|
||
## Repo state
|
||
|
||
### Git
|
||
|
||
- Branch: `main`
|
||
- Remote: `origin` → `https://gitea.doesworks.net/chris.tonjes/YPMS.git` (HTTPS + Gitea access token)
|
||
- `main` tracks `origin/main`. Push with `git -C /Users/christonjes/cc/ypms push`.
|
||
- Milestone history: `git log --oneline` — one "Phase 0 milestone 0.N" commit per milestone, followed by a "Mark milestone 0.N complete" docs commit.
|
||
|
||
### Local git identity (repo-local, not global)
|
||
|
||
```
|
||
Chris Tonjes <ctonjes@gmail.com>
|
||
```
|
||
|
||
If commits should attribute to a DOES email instead, run:
|
||
```bash
|
||
git -C /Users/christonjes/cc/ypms config user.email <addr>
|
||
```
|
||
|
||
### Test / coverage
|
||
|
||
| Component | Coverage |
|
||
|---|---|
|
||
| `framework/` (base, bulk, crud, pagination, pii_audit, version_history) | 91–100% |
|
||
| `security/` (models, rbac) | 80–100% |
|
||
| `auth/` (security, deps, router, schemas, rate_limit) | 80–100% |
|
||
| `seed/` (roles, permissions, run) | 77–100% |
|
||
| `youth/` (models, router, schemas) | 99–100% |
|
||
| `security/column_crypto.py` | 100% |
|
||
| `framework/stub_router.py`, `stubs.py` | 100% |
|
||
| `framework/retention.py`, `retention_classes.py`, `stub_models.py` | 99–100% |
|
||
| Overall | **95.83%** |
|
||
|
||
`framework/router.py` is still excluded from coverage — Registrant uses a hand-built router, so `make_router` gets its first real callers in 0.6. Coverage config in `pyproject.toml`.
|
||
|
||
---
|
||
|
||
## How to run locally
|
||
|
||
### Docker (canonical)
|
||
|
||
```bash
|
||
cd /Users/christonjes/cc/ypms
|
||
cp .env.example .env
|
||
docker compose up --build
|
||
```
|
||
|
||
- App: http://localhost:8000 (OpenAPI at `/docs`)
|
||
- Adminer: http://localhost:8081 (server `db`, user/pw/db all `ypms`)
|
||
- On first boot, seeding runs and `chris` is created with password `change-me-immediately` (or `SEED_ADMIN_PASSWORD` env). Change it via `POST /auth/change-password`.
|
||
|
||
### Native (no Docker)
|
||
|
||
```bash
|
||
cd /Users/christonjes/cc/ypms
|
||
python3 -m venv .venv && source .venv/bin/activate
|
||
pip install -e '.[dev]'
|
||
pytest
|
||
```
|
||
|
||
Tests use SQLite in-memory — no Postgres required.
|
||
|
||
### Log in from the CLI
|
||
|
||
```bash
|
||
curl -X POST http://localhost:8000/auth/login \
|
||
-H 'Content-Type: application/x-www-form-urlencoded' \
|
||
-d 'username=chris&password=change-me-immediately'
|
||
```
|
||
|
||
Response includes `access_token`. Use it as `Authorization: Bearer <token>`.
|
||
|
||
---
|
||
|
||
## How to verify a milestone before committing
|
||
|
||
```bash
|
||
source .venv/bin/activate
|
||
ruff check src tests # style
|
||
mypy src # types (strict)
|
||
pytest -q --cov=src/ypms --cov-report=term --cov-fail-under=80
|
||
```
|
||
|
||
Before pushing, also run the remaining CI gates:
|
||
|
||
```bash
|
||
bandit -q -r src -c pyproject.toml
|
||
pip-audit --strict . # runtime dependency set from pyproject
|
||
```
|
||
|
||
The Gitea Actions workflow at `.gitea/workflows/ci.yml` runs all of the above plus gitleaks and the SBOM build.
|
||
|
||
---
|
||
|
||
## Gotchas — things that would trip up someone new
|
||
|
||
1. **`app = create_app()` at the bottom of `src/ypms/main.py` runs at import time.** Tests must have env vars set before that module is imported. `tests/conftest.py` does this at *module* level (not in a fixture) — do not move it back into a fixture.
|
||
|
||
2. **SQLite in-memory tests need `StaticPool` + `check_same_thread=False`.** `TestClient` runs FastAPI handlers in a worker thread. The `db` fixture in `tests/conftest.py` configures the engine correctly — don't strip those options.
|
||
|
||
3. **`framework/router.py` deliberately does NOT use `from __future__ import annotations`.** FastAPI introspects route parameter types via `typing.get_type_hints`, which would fail to resolve closure-local TypeVars under PEP 563. See the module docstring.
|
||
|
||
4. **Two mypy overrides in `pyproject.toml`** — `framework/router.py` (FastAPI closure typevars) and `framework/crud.py` (generic-CRUD TypeVar attribute access). Same limitations OWC accepts.
|
||
|
||
5. **The vendored OWC framework had a pagination bug** — `next_cursor` was the peek-ahead row's id, silently dropping one row per page boundary. **Fixed in YPMS's copy** at `framework/crud.py`. If OWC is re-vendored, apply the fix again.
|
||
|
||
6. **`DOES_SYSTEM_ADMIN` does NOT appear in `permissions_matrix.json` under `grants`.** It bypasses via `security.rbac.ADMIN_ROLE`. Adding rows for it would be redundant and confuse the "who really has what" story.
|
||
|
||
7. **Row-level scoping is a Phase 2+ concern.** `HOST_USER` has `Host.Edit` in the matrix, which means the RBAC check passes. The service layer must ALSO filter by ownership before applying the edit. Don't rely on the matrix alone for row-level authorization.
|
||
|
||
8. **Alembic baseline (0001) is empty by design.** All schema lives in 0002+ so that "the framework tables" can be re-derived as an additive layer during future architecture reviews.
|
||
|
||
9. **`Registrant.ssn` has no getter.** Reading it raises `AttributeError` on purpose. Use `decrypt_ssn()` for the one sanctioned plaintext path, and never put it in a response schema or log line.
|
||
|
||
10. **Any new encrypted column must be listed in the model's `__audit_redact__`** — otherwise its ciphertext lands in `audit_log.before/after` and `version_history.snapshot`.
|
||
|
||
11. **`COLUMN_ENCRYPTION_KEY` changes make existing ciphertext unreadable.** Phase 0 has no rotation; don't change the dev key on a database you want to keep. `column_crypto._aead` is `lru_cache`d — tests that change the key must `cache_clear()` it and `get_settings`.
|
||
|
||
12. **The `api` test fixture shares one app across the session.** Tests must not mutate app state other than `dependency_overrides[get_db]` (which the fixture resets). If a test needs custom routes or settings, build its own app with `create_app()`.
|
||
|
||
13. **Adding an entity to `permissions_matrix.json` fails `test_every_matrix_entity_is_real_or_stubbed`** until it's either stubbed in `stubs.py` or has a real router (then add it to `REAL_ENTITIES` in that test).
|
||
|
||
14. **Every mapped table must resolve a retention policy** — including test-only models (`tests/framework/_fixtures.py::Thing` declares one). A new model without `retention_policy` fails `test_every_mapped_table_has_a_policy` and makes the job raise `LookupError`.
|
||
|
||
15. **New model modules go in `src/ypms/models.py`.** Alembic, the retention job, and the test DB all read that list.
|
||
|
||
16. **`JWT_SECRET` must be ≥ 32 bytes or the app won't start** (`Settings` validation). Old `.env` files with `dev-secret-change-me` need updating.
|
||
|
||
17. **JWT library is PyJWT, not python-jose** (diverges from OWC). Catch `jwt.PyJWTError`, not `JWTError`.
|
||
|
||
---
|
||
|
||
## File map
|
||
|
||
Read in this order if you're picking this up cold:
|
||
|
||
1. [`CLAUDE.md`](CLAUDE.md) — project router, distinctions, open questions, resolved decisions
|
||
2. [`OVERVIEW.md`](OVERVIEW.md) — programs served, users, integrations, timeline
|
||
3. [`ARCHITECTURE.md`](ARCHITECTURE.md) — design blueprint (§4 rationale, §5 layers, §6 data, §12 delivery phases)
|
||
4. [`PHASE0_PLAN.md`](PHASE0_PLAN.md) — phase scope, milestone exit criteria, controls-wired-in-Phase-0
|
||
5. **This file** — where we are + what's next + gotchas
|
||
6. [`README.md`](README.md) — runnable quickstart
|
||
7. [`CONTRIBUTING.md`](CONTRIBUTING.md) — hard rules
|
||
|
||
Then the ATO artifacts as needed:
|
||
|
||
- [`SYSTEM_SECURITY_PLAN.md`](SYSTEM_SECURITY_PLAN.md) v0.3 — SSP
|
||
- [`NIST_800_53_CONTROL_MAPPING.md`](NIST_800_53_CONTROL_MAPPING.md) v0.2 — control mapping
|
||
- [`PRIVACY_IMPACT_ASSESSMENT.md`](PRIVACY_IMPACT_ASSESSMENT.md) v0.1 — PIA
|
||
- [`POAM.md`](POAM.md) v0.1 — 62 items, 2 closed, 4 partially closed
|
||
- [`OLA_RESPONSE_2026-09-30.md`](OLA_RESPONSE_2026-09-30.md) — external Q&A snapshot
|
||
|
||
---
|
||
|
||
## POA&M state (as of 2026-10-01)
|
||
|
||
- **62 items total** — 2 fully closed; 9 Phase 0 milestone partial closures logged under `YPMS-POAM-040`; `YPMS-POAM-042` (CI/SBOM) in progress pending the runner
|
||
- **6 open decisions** (see Open Questions table above)
|
||
- **Immediate remaining action items:**
|
||
- PIA route to Privacy Officer + OGC review
|
||
- Pull OSSE / CFSA / SSA existing agreements from OGC files to scope renewals
|
||
- Draft FIPS 199 categorization memo (`YPMS-POAM-001`) for AO signature
|
||
- Schedule DC OCTO security architecture review
|
||
|
||
---
|
||
|
||
## Signals to watch for at Phase 0 exit / Phase 1 start
|
||
|
||
- If the first real CI run fails on something local runs didn't catch (runner image, network), fix the workflow, not the gate.
|
||
- Phase 1 promotes Guardian and ConsentRecord from stubs: follow README "Adding an entity", and give both the Registrant treatment (`pii_read`, `__audit_redact__` for anything encrypted, coverage doc).
|
||
- PARENT_USER access to a child's Registrant hangs on the Guardian link — replace the Phase 0 "parents see nothing" rule in `youth/router.py`, don't layer around it.
|
||
- Keep `docs/nist-mapping.md` and the control-mapping rows in step; line numbers drift, so refresh them at each phase gate.
|
||
- Retention execution (actual disposal) is Phase 5 work, alongside the OSSE/CFSA ingest pipelines. Don't add it in passing.
|
||
---
|
||
|
||
**End of handoff v0.6.**
|