Phase 0 milestone 0.1: repo scaffold + Docker Compose
Scaffolding for the YPMS in-house build. Mirrors OWC's structure so future OWC framework code can be vendored in with minimal fiddling (milestone 0.2). Includes: - pyproject.toml — Python 3.12, FastAPI, SQLAlchemy 2.x, Alembic, Postgres, pydantic-settings, structlog, cryptography. Dev extras: pytest, ruff, mypy, bandit, pip-audit. - Dockerfile — multi-stage builder + slim runtime. - docker-compose.yml — app + Postgres (on 5433 to avoid clashing with OWC on 5432) + Adminer (on 8081). - alembic.ini + migrations/env.py + baseline 0001_baseline.py (empty by design; every future change is additive and immutable once merged). - src/ypms/main.py — FastAPI factory with /healthz and /readyz. - src/ypms/settings.py — pydantic-settings config (DB URL, JWT, column encryption key, log level, env, CORS). - src/ypms/logging.py — structlog + sanitize_for_log() for the sensitive-fields-never-in-logs rule from ARCHITECTURE.md §10.1. - src/ypms/db.py — SQLAlchemy Base, engine, session factory, get_db. - src/ypms/middleware.py — RequestIdMiddleware; X-Request-ID header echoed both ways. - src/ypms/framework/ and src/ypms/security/ — empty packages, to be populated in milestones 0.2 and 0.3-0.4. - tests/test_health.py — 3 smoke tests (healthz OK, header present, header honoured on incoming). - README.md — quickstart. - CONTRIBUTING.md — hard rules mirrored from OWC. - .github/workflows/ci.yml — lint (ruff), types (mypy), tests + 80% coverage floor, SAST (bandit), SCA (pip-audit), gitleaks. Green locally: ruff clean, mypy clean, pytest 3 passed. Advances YPMS-POAM-040 (architecture doc + Phase 0 kickoff). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
8712e93258
commit
393631ad69
23 files changed
+851
No files matched your search
@@ -0,0 +1,21 @@
|
||||
.git
|
||||
.gitignore
|
||||
.github
|
||||
.venv
|
||||
venv
|
||||
__pycache__
|
||||
*.pyc
|
||||
.pytest_cache
|
||||
.mypy_cache
|
||||
.ruff_cache
|
||||
.coverage
|
||||
htmlcov
|
||||
*.md
|
||||
docs
|
||||
tests
|
||||
.env
|
||||
.env.*
|
||||
.DS_Store
|
||||
.vscode
|
||||
.idea
|
||||
pg_data
|
||||
@@ -0,0 +1,27 @@
|
||||
# YPMS local development environment
|
||||
#
|
||||
# Copy this file to .env for local runs. Do NOT commit .env.
|
||||
# Production values live in Azure Key Vault (Phase 1+).
|
||||
|
||||
# ---- Database ----
|
||||
# Local (host) → Postgres exposed on 5433 to avoid clashing with OWC on 5432
|
||||
DATABASE_URL=postgresql+psycopg://ypms:ypms@localhost:5433/ypms
|
||||
|
||||
# ---- JWT ----
|
||||
JWT_SECRET=dev-secret-change-me
|
||||
JWT_ALG=HS256
|
||||
ACCESS_TOKEN_TTL_MINUTES=60
|
||||
REFRESH_TOKEN_TTL_DAYS=14
|
||||
|
||||
# ---- App-layer column encryption ----
|
||||
# Phase 0: local key material only. Phase 1 swaps to Azure Key Vault.
|
||||
# Any 32-byte value in dev. Never use this string in staging/prod.
|
||||
COLUMN_ENCRYPTION_KEY=dev-col-enc-key-change-me-32bytes-______
|
||||
|
||||
# ---- Logging + environment ----
|
||||
LOG_LEVEL=INFO
|
||||
ENV=dev
|
||||
|
||||
# ---- CORS (dev only) ----
|
||||
# Comma-separated origins allowed to hit the API from a browser.
|
||||
CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
|
||||
@@ -0,0 +1,47 @@
|
||||
name: ci
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
test:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python 3.12
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
cache: pip
|
||||
|
||||
- name: Install
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -e '.[dev]'
|
||||
|
||||
- name: Lint (ruff)
|
||||
run: ruff check src tests
|
||||
|
||||
- name: Type check (mypy)
|
||||
run: mypy src
|
||||
|
||||
- name: Tests + coverage
|
||||
run: pytest --cov=src/ypms --cov-report=term-missing --cov-fail-under=80
|
||||
|
||||
- name: SAST (bandit)
|
||||
run: bandit -r src -c pyproject.toml
|
||||
|
||||
- name: SCA (pip-audit)
|
||||
run: pip-audit --strict --disable-pip
|
||||
|
||||
- name: Gitleaks
|
||||
uses: gitleaks/gitleaks-action@v2
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
@@ -0,0 +1,52 @@
|
||||
# Contributing to YPMS
|
||||
|
||||
Hard rules — mirrored from OWC's `CONTRIBUTING.md` because the two systems share a stack, an ownership model, and an ISSO. If a rule feels wrong for YPMS, propose an amendment in a PR to this file, don't quietly break it.
|
||||
|
||||
## Repository conventions
|
||||
|
||||
1. **Migrations are immutable once merged.** Add a new Alembic revision, never edit `0001_baseline.py` or its successors. If a schema change requires undoing an older migration, the fix is a new revision, not a rewrite. Every migration is a paper trail an auditor can walk.
|
||||
|
||||
2. **Don't add domain logic to `src/ypms/framework/` or `src/ypms/security/`.** Those are reused by every entity; entity-specific code belongs in `src/ypms/<entity>/`. If a framework enhancement is genuinely needed, propose it in its own PR with a written reason.
|
||||
|
||||
3. **Every state-mutating route emits an audit row.** Enforced by `CRUDService` and the router factory once the framework lands. Don't write handlers that bypass either.
|
||||
|
||||
4. **501 stubs still fire RBAC + audit.** Deferred actions must route through `require_permission` and emit an audit event. Permission gaps surface at Phase 0 review, not at feature-completion.
|
||||
|
||||
5. **Tests required.** `src/ypms/framework/` and `src/ypms/security/` need ≥ 80% coverage. Everything else: at least one happy-path test.
|
||||
|
||||
6. **No `--no-verify`, `# noqa`, or `# type: ignore` without an inline comment naming the specific reason.** If you need to silence a check, say why in the code.
|
||||
|
||||
7. **Sensitive fields never in logs.** SSN, banking, tokens, passwords, encryption keys — route through `sanitize_for_log` before emitting or don't emit. See `src/ypms/logging.py`.
|
||||
|
||||
8. **No commits attributed to a person who didn't write them.** Set `user.name` and `user.email` in `git config` to yourself before committing. Co-author trailers are welcome and encouraged for pairing.
|
||||
|
||||
## Change control
|
||||
|
||||
- Every substantive change ties to a POA&M ID, a phase-plan milestone, or an issue. Include the reference in the PR description.
|
||||
- `PHASE{N}_PLAN.md` is authoritative for the current phase's scope. Older plans lose to the current one on conflict.
|
||||
- Cross-phase work needs a written scope-conflict note before it lands.
|
||||
|
||||
## Security review
|
||||
|
||||
- Any change touching auth, RBAC, encryption, or audit gets a second reviewer.
|
||||
- Changes to `src/ypms/security/` require the ISSO's approval (currently Chris Tonjes).
|
||||
- Additions to the RBAC seed require the same review as code changes — permissions are policy in this codebase.
|
||||
|
||||
## PR checklist
|
||||
|
||||
- [ ] Tests pass locally (`pytest`).
|
||||
- [ ] Coverage doesn't drop below 80% on `framework/` or `security/`.
|
||||
- [ ] `ruff check`, `mypy` clean.
|
||||
- [ ] SAST (`bandit`) and SCA (`pip-audit`) clean, or documented waivers.
|
||||
- [ ] Migration added (if schema changed); no edits to older migrations.
|
||||
- [ ] Any new endpoint has RBAC + audit + a test.
|
||||
- [ ] Any new sensitive field routes through the app-layer encryption helpers.
|
||||
- [ ] `docs/nist-mapping.md` updated if the change wires a new NIST control.
|
||||
- [ ] POA&M item updated if the change closes or advances one.
|
||||
- [ ] Commit message names the phase milestone or POA&M ID it advances.
|
||||
|
||||
## Merging
|
||||
|
||||
- Squash-merge is the default. Merge commit messages should read like release notes for that PR.
|
||||
- Force-pushes to `main` are not allowed. Force-pushes to feature branches during review are fine.
|
||||
- CI must be green.
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
FROM python:3.12-slim AS builder
|
||||
WORKDIR /app
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends build-essential && rm -rf /var/lib/apt/lists/*
|
||||
COPY pyproject.toml ./
|
||||
COPY src ./src
|
||||
RUN pip install --no-cache-dir --prefix=/install .
|
||||
|
||||
FROM python:3.12-slim AS runtime
|
||||
WORKDIR /app
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends libpq5 && rm -rf /var/lib/apt/lists/*
|
||||
COPY --from=builder /install /usr/local
|
||||
COPY src ./src
|
||||
COPY migrations ./migrations
|
||||
COPY alembic.ini ./
|
||||
ENV PYTHONPATH=/app/src
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
EXPOSE 8000
|
||||
CMD ["uvicorn", "ypms.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
@@ -0,0 +1,83 @@
|
||||
# YPMS — Youth Program Management System
|
||||
|
||||
DC Department of Employment Services / Office of Youth Programs — in-house build replacing seven legacy tools (MBSYEP intake, TMS timekeeping, PF3 admin, manual payroll scripts, MicroStrategy, and two supporting apps) with a single Python / FastAPI / Postgres system on Azure Government.
|
||||
|
||||
**Status:** Pre-development. Phase 0 (foundations) in flight.
|
||||
**System owner + ISSO + AO:** Chris Tonjes, DOES CIO.
|
||||
**Full context:** [`OVERVIEW.md`](OVERVIEW.md), [`ARCHITECTURE.md`](ARCHITECTURE.md), [`PHASE0_PLAN.md`](PHASE0_PLAN.md).
|
||||
|
||||
## Quickstart (local)
|
||||
|
||||
Requires Docker Desktop (or equivalent). Ports 8000 (app), 5433 (Postgres), 8081 (Adminer) must be free.
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
App at http://localhost:8000. OpenAPI docs at http://localhost:8000/docs.
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/healthz # → {"status":"ok"}
|
||||
curl http://localhost:8000/readyz # → {"status":"ok"} once db is up
|
||||
```
|
||||
|
||||
Adminer (DB explorer) at http://localhost:8081. Connect with:
|
||||
- System: PostgreSQL
|
||||
- Server: `db`
|
||||
- Username / Password / Database: `ypms` / `ypms` / `ypms`
|
||||
|
||||
Stop with `Ctrl+C`; wipe local DB with `docker compose down -v`.
|
||||
|
||||
## Tests
|
||||
|
||||
Local (no Docker required for the tests):
|
||||
|
||||
```bash
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip install -e '.[dev]'
|
||||
pytest
|
||||
```
|
||||
|
||||
Coverage:
|
||||
|
||||
```bash
|
||||
pytest --cov=src/ypms --cov-report=term-missing
|
||||
```
|
||||
|
||||
Tests use SQLite in-memory for speed. Postgres-specific behaviour gets its own Postgres-only fixture when a migration or query needs it (see [`ARCHITECTURE.md`](ARCHITECTURE.md) §4.8).
|
||||
|
||||
## Repo shape
|
||||
|
||||
Full tree in [`ARCHITECTURE.md`](ARCHITECTURE.md) §5.1. High points:
|
||||
|
||||
- `src/ypms/main.py` — FastAPI factory
|
||||
- `src/ypms/settings.py` — pydantic-settings config
|
||||
- `src/ypms/db.py` — SQLAlchemy engine, session, `Base`
|
||||
- `src/ypms/logging.py` — structlog + sensitive-field sanitizer
|
||||
- `src/ypms/middleware.py` — request id
|
||||
- `src/ypms/framework/` — vendored from OWC, milestone 0.2
|
||||
- `src/ypms/security/` — auth + RBAC + column encryption, milestones 0.3–0.4
|
||||
- `src/ypms/youth/` — Registrant entity (vertical slice), milestone 0.5
|
||||
- Other entity packages — 501 stubs, milestone 0.6
|
||||
- `migrations/` — Alembic, immutable-once-merged
|
||||
- `tests/` — pytest, SQLite by default
|
||||
|
||||
## Adding an entity
|
||||
|
||||
Once the framework lands in milestone 0.2, adding a new entity follows the OWC add-entity pattern (model → schemas → router → main.py registration → migration → permissions seed entry → test → coverage doc). Full instructions land here at milestone 0.2 completion.
|
||||
|
||||
## Contributing
|
||||
|
||||
See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the hard rules. Highlights: migrations are immutable once merged, every state-mutating route emits an audit row, framework/security modules must not carry domain logic, no `--no-verify` or `# type: ignore` without a written reason.
|
||||
|
||||
## Compliance
|
||||
|
||||
- [`SYSTEM_SECURITY_PLAN.md`](SYSTEM_SECURITY_PLAN.md) v0.3 — NIST 800-53 Moderate SSP
|
||||
- [`NIST_800_53_CONTROL_MAPPING.md`](NIST_800_53_CONTROL_MAPPING.md) v0.2 — full control-family map
|
||||
- [`PRIVACY_IMPACT_ASSESSMENT.md`](PRIVACY_IMPACT_ASSESSMENT.md) v0.1 — PIA covering minors, SSN, banking, FERPA, CFSA
|
||||
- [`POAM.md`](POAM.md) v0.1 — POA&M with 62 items and 2 closures as of 2026-09-30
|
||||
|
||||
## License / usage
|
||||
|
||||
Internal DOES system. Not for external distribution.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
[alembic]
|
||||
script_location = migrations
|
||||
prepend_sys_path = src
|
||||
version_path_separator = os
|
||||
sqlalchemy.url =
|
||||
|
||||
[loggers]
|
||||
keys = root,sqlalchemy,alembic
|
||||
|
||||
[handlers]
|
||||
keys = console
|
||||
|
||||
[formatters]
|
||||
keys = generic
|
||||
|
||||
[logger_root]
|
||||
level = WARN
|
||||
handlers = console
|
||||
qualname =
|
||||
|
||||
[logger_sqlalchemy]
|
||||
level = WARN
|
||||
handlers =
|
||||
qualname = sqlalchemy.engine
|
||||
|
||||
[logger_alembic]
|
||||
level = INFO
|
||||
handlers =
|
||||
qualname = alembic
|
||||
|
||||
[handler_console]
|
||||
class = StreamHandler
|
||||
args = (sys.stderr,)
|
||||
level = NOTSET
|
||||
formatter = generic
|
||||
|
||||
[formatter_generic]
|
||||
format = %(levelname)-5.5s [%(name)s] %(message)s
|
||||
datefmt = %H:%M:%S
|
||||
@@ -0,0 +1,50 @@
|
||||
services:
|
||||
db:
|
||||
image: postgres:16
|
||||
environment:
|
||||
POSTGRES_USER: ypms
|
||||
POSTGRES_PASSWORD: ypms
|
||||
POSTGRES_DB: ypms
|
||||
TZ: UTC
|
||||
PGTZ: UTC
|
||||
volumes:
|
||||
- pg_data:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "5433:5432"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ypms"]
|
||||
interval: 3s
|
||||
timeout: 3s
|
||||
retries: 20
|
||||
|
||||
app:
|
||||
build: .
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
environment:
|
||||
DATABASE_URL: postgresql+psycopg://ypms:ypms@db:5432/ypms
|
||||
JWT_SECRET: dev-secret-change-me
|
||||
JWT_ALG: HS256
|
||||
ACCESS_TOKEN_TTL_MINUTES: 60
|
||||
COLUMN_ENCRYPTION_KEY: dev-col-enc-key-change-me-32bytes-______
|
||||
LOG_LEVEL: INFO
|
||||
ENV: dev
|
||||
ports:
|
||||
- "8000:8000"
|
||||
command: >
|
||||
sh -c "alembic upgrade head &&
|
||||
uvicorn ypms.main:app --host 0.0.0.0 --port 8000 --reload"
|
||||
volumes:
|
||||
- ./src:/app/src
|
||||
- ./migrations:/app/migrations
|
||||
|
||||
adminer:
|
||||
image: adminer:4
|
||||
depends_on:
|
||||
- db
|
||||
ports:
|
||||
- "8081:8080"
|
||||
|
||||
volumes:
|
||||
pg_data:
|
||||
@@ -0,0 +1,55 @@
|
||||
"""Alembic environment.
|
||||
|
||||
At milestone 0.1 the metadata is empty (no models yet). Milestones 0.5 and 0.6
|
||||
register real and stub entity models, at which point `target_metadata` will
|
||||
carry their tables. `alembic upgrade head` currently applies a no-op baseline.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from logging.config import fileConfig
|
||||
|
||||
from alembic import context
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
|
||||
from ypms.db import Base
|
||||
from ypms.settings import get_settings
|
||||
|
||||
# ---- Import model modules here so Base.metadata is populated ----
|
||||
# (empty at milestone 0.1; entities register themselves via later milestones)
|
||||
|
||||
config = context.config
|
||||
if config.config_file_name is not None:
|
||||
fileConfig(config.config_file_name)
|
||||
|
||||
config.set_main_option("sqlalchemy.url", get_settings().database_url)
|
||||
target_metadata = Base.metadata
|
||||
|
||||
|
||||
def run_migrations_offline() -> None:
|
||||
context.configure(
|
||||
url=config.get_main_option("sqlalchemy.url"),
|
||||
target_metadata=target_metadata,
|
||||
literal_binds=True,
|
||||
dialect_opts={"paramstyle": "named"},
|
||||
)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
def run_migrations_online() -> None:
|
||||
connectable = engine_from_config(
|
||||
config.get_section(config.config_ini_section, {}),
|
||||
prefix="sqlalchemy.",
|
||||
poolclass=pool.NullPool,
|
||||
)
|
||||
with connectable.connect() as connection:
|
||||
context.configure(connection=connection, target_metadata=target_metadata)
|
||||
with context.begin_transaction():
|
||||
context.run_migrations()
|
||||
|
||||
|
||||
if context.is_offline_mode():
|
||||
run_migrations_offline()
|
||||
else:
|
||||
run_migrations_online()
|
||||
@@ -0,0 +1,27 @@
|
||||
"""${message}
|
||||
|
||||
Revision ID: ${up_revision}
|
||||
Revises: ${down_revision | comma,n}
|
||||
Create Date: ${create_date}
|
||||
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
${imports if imports else ""}
|
||||
|
||||
revision: str = ${repr(up_revision)}
|
||||
down_revision: str | Sequence[str] | None = ${repr(down_revision)}
|
||||
branch_labels: str | Sequence[str] | None = ${repr(branch_labels)}
|
||||
depends_on: str | Sequence[str] | None = ${repr(depends_on)}
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
${upgrades if upgrades else "pass"}
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
${downgrades if downgrades else "pass"}
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Baseline — empty schema.
|
||||
|
||||
Revision ID: 0001_baseline
|
||||
Revises:
|
||||
Create Date: 2026-09-30
|
||||
|
||||
Milestone 0.1 baseline. Deliberately empty — proves that `alembic upgrade head`
|
||||
runs against a fresh database. Real tables land as additive migrations in
|
||||
milestones 0.5 (Registrant) and 0.6 (stub-entity metadata).
|
||||
|
||||
Per repo rule (CONTRIBUTING.md): this file is immutable once merged. Every
|
||||
subsequent change is a new revision, never an edit here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
|
||||
revision: str = "0001_baseline"
|
||||
down_revision: str | Sequence[str] | None = None
|
||||
branch_labels: str | Sequence[str] | None = None
|
||||
depends_on: str | Sequence[str] | None = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# Intentionally empty. See module docstring.
|
||||
pass
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Nothing to undo.
|
||||
pass
|
||||
@@ -0,0 +1,69 @@
|
||||
[project]
|
||||
name = "ypms"
|
||||
version = "0.1.0"
|
||||
description = "DC Youth Program Management System"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = [
|
||||
"fastapi>=0.115",
|
||||
"uvicorn[standard]>=0.30",
|
||||
"sqlalchemy>=2.0",
|
||||
"alembic>=1.13",
|
||||
"psycopg[binary]>=3.2",
|
||||
"pydantic[email]>=2.8",
|
||||
"pydantic-settings>=2.4",
|
||||
"passlib>=1.7",
|
||||
"python-jose[cryptography]>=3.3",
|
||||
"python-multipart>=0.0.9",
|
||||
"structlog>=24.4",
|
||||
"cryptography>=43",
|
||||
"jinja2>=3.1",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8.3",
|
||||
"pytest-asyncio>=0.24",
|
||||
"pytest-cov>=5.0",
|
||||
"httpx>=0.27",
|
||||
"ruff>=0.6",
|
||||
"mypy>=1.11",
|
||||
"bandit>=1.7",
|
||||
"pip-audit>=2.7",
|
||||
]
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
asyncio_mode = "auto"
|
||||
addopts = "-ra --strict-markers"
|
||||
|
||||
[tool.coverage.run]
|
||||
source = ["src/ypms"]
|
||||
omit = ["tests/*"]
|
||||
|
||||
[tool.coverage.report]
|
||||
show_missing = true
|
||||
skip_covered = false
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
target-version = "py312"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "I", "UP", "B", "SIM"]
|
||||
# E501: handled by formatter; B008: FastAPI's Depends() in defaults is canonical
|
||||
ignore = ["E501", "B008"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.12"
|
||||
strict = true
|
||||
ignore_missing_imports = true
|
||||
|
||||
[tool.bandit]
|
||||
exclude_dirs = ["tests", "migrations"]
|
||||
Whitespace-only changes.
@@ -0,0 +1,38 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Iterator
|
||||
|
||||
from sqlalchemy import Engine, create_engine
|
||||
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
|
||||
|
||||
from ypms.settings import get_settings
|
||||
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
pass
|
||||
|
||||
|
||||
_engine: Engine | None = None
|
||||
_SessionLocal: sessionmaker[Session] | None = None
|
||||
|
||||
|
||||
def engine() -> Engine:
|
||||
global _engine
|
||||
if _engine is None:
|
||||
_engine = create_engine(get_settings().database_url, pool_pre_ping=True, future=True)
|
||||
return _engine
|
||||
|
||||
|
||||
def session_factory() -> sessionmaker[Session]:
|
||||
global _SessionLocal
|
||||
if _SessionLocal is None:
|
||||
_SessionLocal = sessionmaker(bind=engine(), autoflush=False, autocommit=False, future=True)
|
||||
return _SessionLocal
|
||||
|
||||
|
||||
def get_db() -> Iterator[Session]:
|
||||
db = session_factory()()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
@@ -0,0 +1,6 @@
|
||||
"""YPMS framework package.
|
||||
|
||||
Phase 0 milestone 0.2 will vendor in OWC's `framework/` modules (base entity,
|
||||
mixins, CRUDService, router factory, audit emitter, pagination) and add YPMS's
|
||||
`pii_audit.py` decorator on top. This package is empty at milestone 0.1.
|
||||
"""
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Structured logging setup.
|
||||
|
||||
JSON logs in non-dev environments so Sentinel can ingest them cleanly. Console
|
||||
renderer in dev for readability.
|
||||
|
||||
Sensitive fields (SSN, banking account numbers, tokens, passwords) MUST NEVER
|
||||
be logged. The `sanitize_for_log` helper strips them before emit; use it whenever
|
||||
you log a dict that could contain PII. Callers that pass raw request bodies are
|
||||
a Phase 0 bug — fix them, don't paper over with a broader sanitizer here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from typing import Any
|
||||
|
||||
import structlog
|
||||
|
||||
from ypms.settings import get_settings
|
||||
|
||||
_SENSITIVE_KEYS = {
|
||||
"ssn",
|
||||
"ssn_encrypted",
|
||||
"password",
|
||||
"password_hash",
|
||||
"secret",
|
||||
"token",
|
||||
"access_token",
|
||||
"refresh_token",
|
||||
"authorization",
|
||||
"banking_account_number",
|
||||
"routing_number",
|
||||
"column_encryption_key",
|
||||
"jwt_secret",
|
||||
}
|
||||
|
||||
|
||||
def configure_logging() -> None:
|
||||
settings = get_settings()
|
||||
level = getattr(logging, settings.log_level.upper(), logging.INFO)
|
||||
logging.basicConfig(format="%(message)s", stream=sys.stdout, level=level)
|
||||
|
||||
processors: list[Any] = [
|
||||
structlog.contextvars.merge_contextvars,
|
||||
structlog.processors.add_log_level,
|
||||
structlog.processors.TimeStamper(fmt="iso"),
|
||||
]
|
||||
if settings.env == "dev":
|
||||
processors.append(structlog.dev.ConsoleRenderer())
|
||||
else:
|
||||
processors.append(structlog.processors.JSONRenderer())
|
||||
|
||||
structlog.configure(
|
||||
processors=processors,
|
||||
wrapper_class=structlog.make_filtering_bound_logger(level),
|
||||
logger_factory=structlog.stdlib.LoggerFactory(),
|
||||
cache_logger_on_first_use=True,
|
||||
)
|
||||
|
||||
|
||||
def get_logger(name: str | None = None) -> structlog.stdlib.BoundLogger:
|
||||
return structlog.get_logger(name) # type: ignore[no-any-return] # structlog stubs return Any
|
||||
|
||||
|
||||
def sanitize_for_log(data: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Return a shallow copy of `data` with sensitive keys redacted.
|
||||
|
||||
Callers that log request bodies, DB row dumps, or JWT contents should route
|
||||
through this first. Recursive on dicts; leaves lists alone (add case-by-case
|
||||
as needed).
|
||||
"""
|
||||
out: dict[str, Any] = {}
|
||||
for k, v in data.items():
|
||||
if k.lower() in _SENSITIVE_KEYS:
|
||||
out[k] = "REDACTED"
|
||||
elif isinstance(v, dict):
|
||||
out[k] = sanitize_for_log(v)
|
||||
else:
|
||||
out[k] = v
|
||||
return out
|
||||
@@ -0,0 +1,68 @@
|
||||
"""FastAPI application factory.
|
||||
|
||||
Phase 0 milestone 0.1: minimal shell that boots against Docker Compose, serves
|
||||
`/healthz` (liveness) and `/readyz` (dependency-aware readiness), and emits
|
||||
structured logs with a request id per request. Domain routers get added in
|
||||
subsequent milestones.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import FastAPI, status
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from sqlalchemy import text
|
||||
|
||||
from ypms.db import engine
|
||||
from ypms.logging import configure_logging, get_logger
|
||||
from ypms.middleware import RequestIdMiddleware
|
||||
from ypms.settings import get_settings
|
||||
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
configure_logging()
|
||||
log = get_logger(__name__)
|
||||
settings = get_settings()
|
||||
|
||||
app = FastAPI(
|
||||
title="YPMS — Youth Program Management System",
|
||||
version="0.1.0",
|
||||
description="DC DOES Office of Youth Programs — in-house build (Phase 0).",
|
||||
)
|
||||
|
||||
# Middleware order: RequestId first so CORS responses carry the id too.
|
||||
app.add_middleware(RequestIdMiddleware)
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=list(settings.cors_origins),
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
expose_headers=["X-Request-ID"],
|
||||
)
|
||||
|
||||
@app.get("/healthz", tags=["system"])
|
||||
def healthz() -> dict[str, str]:
|
||||
"""Liveness — process is up. No dependency checks."""
|
||||
return {"status": "ok"}
|
||||
|
||||
@app.get("/readyz", tags=["system"])
|
||||
def readyz() -> dict[str, str]:
|
||||
"""Readiness — dependencies reachable. Fails 503 if the DB is unreachable."""
|
||||
try:
|
||||
with engine().connect() as conn:
|
||||
conn.execute(text("SELECT 1"))
|
||||
except Exception as e:
|
||||
log.warning("readyz.db_unreachable", error=str(e))
|
||||
from fastapi import HTTPException
|
||||
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail={"status": "db_unreachable"},
|
||||
) from e
|
||||
return {"status": "ok"}
|
||||
|
||||
log.info("app.started", routes=len(app.routes), env=settings.env)
|
||||
return app
|
||||
|
||||
|
||||
app = create_app()
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Cross-cutting middleware.
|
||||
|
||||
- RequestIdMiddleware: attach a short request id to every request, emit it as
|
||||
`X-Request-ID` in responses, and bind it into the structlog contextvars so
|
||||
JSON logs get a `req_id` field automatically for the duration of the request.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
|
||||
import structlog
|
||||
from starlette.middleware.base import BaseHTTPMiddleware
|
||||
from starlette.requests import Request
|
||||
from starlette.responses import Response
|
||||
|
||||
|
||||
class RequestIdMiddleware(BaseHTTPMiddleware):
|
||||
HEADER = "X-Request-ID"
|
||||
|
||||
async def dispatch(self, request: Request, call_next) -> Response: # type: ignore[no-untyped-def]
|
||||
incoming = request.headers.get(self.HEADER)
|
||||
req_id = incoming or uuid.uuid4().hex[:12]
|
||||
request.state.req_id = req_id
|
||||
structlog.contextvars.clear_contextvars()
|
||||
structlog.contextvars.bind_contextvars(req_id=req_id)
|
||||
try:
|
||||
response: Response = await call_next(request)
|
||||
finally:
|
||||
structlog.contextvars.clear_contextvars()
|
||||
response.headers[self.HEADER] = req_id
|
||||
return response
|
||||
@@ -0,0 +1,6 @@
|
||||
"""YPMS security package.
|
||||
|
||||
Phase 0 milestone 0.3 lands JWT + local password auth + the `iam.assurance_level`
|
||||
claim, and milestone 0.4 lands the RBAC engine. This package is empty at
|
||||
milestone 0.1.
|
||||
"""
|
||||
@@ -0,0 +1,35 @@
|
||||
from functools import lru_cache
|
||||
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
|
||||
# Database
|
||||
database_url: str
|
||||
|
||||
# JWT + auth
|
||||
jwt_secret: str
|
||||
jwt_alg: str = "HS256"
|
||||
access_token_ttl_minutes: int = 60
|
||||
refresh_token_ttl_days: int = 14
|
||||
|
||||
# App-layer column encryption
|
||||
# Phase 0: env var. Phase 1: Azure Key Vault.
|
||||
column_encryption_key: str
|
||||
|
||||
# Logging + environment
|
||||
log_level: str = "INFO"
|
||||
env: str = "dev"
|
||||
|
||||
# CORS (dev only — prod overrides)
|
||||
cors_origins: tuple[str, ...] = (
|
||||
"http://localhost:5173",
|
||||
"http://127.0.0.1:5173",
|
||||
)
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
return Settings() # type: ignore[call-arg]
|
||||
Whitespace-only changes.
@@ -0,0 +1,35 @@
|
||||
"""Shared pytest fixtures.
|
||||
|
||||
Uses SQLite for speed (see ARCHITECTURE.md §4.8). Any test that needs
|
||||
Postgres-specific behaviour lives in a Postgres-only fixture path (added when a
|
||||
migration or query needs it).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from collections.abc import Iterator
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def _test_env() -> Iterator[None]:
|
||||
"""Set minimum env vars required for `Settings()` to construct in tests."""
|
||||
os.environ.setdefault("DATABASE_URL", "sqlite:///:memory:")
|
||||
os.environ.setdefault("JWT_SECRET", "test-secret")
|
||||
os.environ.setdefault(
|
||||
"COLUMN_ENCRYPTION_KEY", "test-col-enc-key-32bytes-______________"
|
||||
)
|
||||
os.environ.setdefault("ENV", "test")
|
||||
yield
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client() -> Iterator[TestClient]:
|
||||
from ypms.main import create_app
|
||||
|
||||
app = create_app()
|
||||
with TestClient(app) as c:
|
||||
yield c
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Milestone 0.1 smoke tests.
|
||||
|
||||
Proves:
|
||||
- `/healthz` returns 200 without any dependency access.
|
||||
- Every response carries an `X-Request-ID` header (RequestIdMiddleware wired).
|
||||
- The header is echoed back when the caller supplies one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
|
||||
def test_healthz_returns_ok(client: TestClient) -> None:
|
||||
r = client.get("/healthz")
|
||||
assert r.status_code == 200
|
||||
assert r.json() == {"status": "ok"}
|
||||
|
||||
|
||||
def test_response_carries_request_id(client: TestClient) -> None:
|
||||
r = client.get("/healthz")
|
||||
assert "x-request-id" in {k.lower() for k in r.headers}
|
||||
req_id = r.headers["X-Request-ID"]
|
||||
assert req_id
|
||||
assert len(req_id) >= 8
|
||||
|
||||
|
||||
def test_incoming_request_id_is_honoured(client: TestClient) -> None:
|
||||
r = client.get("/healthz", headers={"X-Request-ID": "test-req-abcd"})
|
||||
assert r.headers["X-Request-ID"] == "test-req-abcd"
|
||||
Reference in new issue
Block a user