310 lines
19 KiB
Markdown
310 lines
19 KiB
Markdown
# Control Panel Operational UX V1 Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Turn the existing server-rendered Control Panel into a bounded operational queue that identifies client/server problems, presents distinct technical/fiscal metrics, and offers safe audited context resolution.
|
|
|
|
**Architecture:** Keep FastAPI, Jinja2, project CSS, and minimal JavaScript. Extend `control_plane.py` with bounded query/view-model functions and add a focused `panel_presentation.py` for timestamps, duration, labels, priorities, and attention ordering. Use existing `FiscalContextEntry`, `Job`, `AuditEvent`, CSRF, and human authorization; do not add remote write capabilities or fiscal-engine changes.
|
|
|
|
**Tech Stack:** Python 3.12, FastAPI, Jinja2, SQLAlchemy, PostgreSQL/SQLite tests, pytest, vanilla CSS/JavaScript.
|
|
|
|
## Global Constraints
|
|
|
|
- Keep FastAPI, Jinja2, CSS próprio, and JavaScript mínimo; do not introduce React, Vue, Node build chain, or SPA.
|
|
- Do not alter fiscal logic, requiredness, Hub/Agent protocol, Firebird writes, AI writes, deployment, Proxmox, or real Agents.
|
|
- `safe_blocked_products` means distinct active tenant/product pairs; `open_blockers` means active blocker rows.
|
|
- Enforce `agents_total == online + degraded + offline + failed + unknown` in code and tests.
|
|
- UI timezone is configured by `UI_TIMEZONE`, default `America/Bahia`; storage/API timestamps remain UTC.
|
|
- Every mutating human action requires human auth, CSRF, deterministic validation, and an `AuditEvent`.
|
|
- Weak local passwords are allowed only with `ALLOW_WEAK_DEV_PASSWORD=1` outside production; production requires the strong policy.
|
|
- Never log or commit credentials, session tokens, passwords, SQL, shell commands, or raw connection strings.
|
|
- Preserve bounded pagination and avoid N+1 queries on 100-installation fixtures.
|
|
|
|
---
|
|
|
|
### Task 1: Presentation Helpers And Password Policy
|
|
|
|
**Files:**
|
|
- Create: `central/app/panel_presentation.py`
|
|
- Modify: `central/app/config.py:16-39`
|
|
- Modify: `central/app/human_auth.py:create_admin`
|
|
- Modify: `central/tests/test_control_plane.py`
|
|
- Create: `central/tests/test_panel_presentation.py`
|
|
- Modify: `docker-compose.yml` to explicitly enable weak development passwords only for the local service
|
|
|
|
**Interfaces:**
|
|
- Produces `format_ui_datetime(value) -> str`, `format_age(value, now=None) -> str`, `format_duration(started_at, finished_at, now=None) -> str`, `operational_priority(blocker) -> str`, and `health_label(value) -> str`.
|
|
- `format_ui_datetime` returns `-` for no value, a localized `dd/mm/YYYY HH:MM` string, and accepts UTC-naive values as UTC.
|
|
- `format_age` returns `-`, `agora`, `há 1 min`, `há N min`, `há 1 h`, or `há N h`; future values return `agora`.
|
|
- `format_duration` returns `-` when no start exists, `em andamento` when only started exists, and compact seconds/minutes/hours when both exist.
|
|
- `create_admin` accepts short passwords only when `settings.environment != "production"` and `ALLOW_WEAK_DEV_PASSWORD=1`; otherwise it retains the strong minimum.
|
|
|
|
- [ ] **Step 1: Write failing timestamp, duration, priority, and password-policy tests**
|
|
|
|
```python
|
|
def test_ui_times_use_configured_timezone_and_human_age(monkeypatch):
|
|
monkeypatch.setenv("UI_TIMEZONE", "America/Bahia")
|
|
value = datetime(2026, 9, 22, 19, 11, tzinfo=timezone.utc)
|
|
assert format_ui_datetime(value) == "22/09/2026 16:11"
|
|
assert format_age(value, datetime(2026, 9, 22, 19, 19, tzinfo=timezone.utc)) == "há 8 min"
|
|
|
|
def test_duration_and_priority_are_deterministic():
|
|
start = datetime(2026, 9, 22, 19, 0, tzinfo=timezone.utc)
|
|
finish = start + timedelta(seconds=194)
|
|
assert format_duration(start, finish) == "3 min 14 s"
|
|
assert operational_priority(SimpleNamespace(blocker_type="TECHNICAL_FAILURE", field="", reason_code="X")) == "CRITICAL"
|
|
|
|
def test_short_local_password_requires_explicit_setting(monkeypatch):
|
|
monkeypatch.delenv("ALLOW_WEAK_DEV_PASSWORD", raising=False)
|
|
with pytest.raises(ValueError):
|
|
create_admin(session, "admin", "admin")
|
|
monkeypatch.setenv("ALLOW_WEAK_DEV_PASSWORD", "1")
|
|
assert create_admin(session, "admin", "admin").username == "admin"
|
|
```
|
|
|
|
- [ ] **Step 2: Run the focused tests and verify they fail**
|
|
|
|
Run: `python -m pytest central/tests/test_panel_presentation.py central/tests/test_control_plane.py -q`
|
|
|
|
Expected: FAIL because the presentation module and environment-aware password gate do not yet exist.
|
|
|
|
- [ ] **Step 3: Implement the helpers and environment-aware password check**
|
|
|
|
Use `zoneinfo.ZoneInfo(settings.ui_timezone)` for display conversion. Add `ui_timezone` and `allow_weak_dev_password` configuration fields, but read the weak-password environment flag at call time so tests and explicit local process settings cannot be stale. Update the local Compose service with `ALLOW_WEAK_DEV_PASSWORD=1`; do not set it for production configuration.
|
|
|
|
- [ ] **Step 4: Run focused tests and inspect the diff**
|
|
|
|
Run: `python -m pytest central/tests/test_panel_presentation.py central/tests/test_control_plane.py -q` and `git diff --check`.
|
|
|
|
Expected: PASS with no whitespace errors and no credential values in the diff.
|
|
|
|
- [ ] **Step 5: Commit the slice**
|
|
|
|
```text
|
|
git add central/app/panel_presentation.py central/app/config.py central/app/human_auth.py central/tests/test_panel_presentation.py central/tests/test_control_plane.py docker-compose.yml
|
|
git commit -m "feat: add operational panel presentation helpers"
|
|
```
|
|
|
|
### Task 2: Bounded Operational Query View-Models
|
|
|
|
**Files:**
|
|
- Modify: `central/app/control_plane.py`
|
|
- Modify: `central/app/models.py` only if a missing relationship-free field is proven necessary; do not add a migration for presentation-only data
|
|
- Modify: `central/tests/test_control_plane.py`
|
|
- Create: `central/tests/test_operational_view_models.py`
|
|
|
|
**Interfaces:**
|
|
- Produces `dashboard_summary(session, now=None)` with `health_counts`, `agents_total`, `safe_blocked_products`, `open_blockers`, `open_blocker_breakdown`, `attention_items`, `failed_jobs`, and `outdated_agents`.
|
|
- Produces `list_installations(..., sort="display_name")` with `fiscal_completion_percent`, `technical_health`, `fiscal_status`, `version_drift`, formatted presentation fields, and `installation_id`/`agent_id` as secondary data.
|
|
- Produces `list_jobs(..., client=None, search=None)` joining `Job` to `Agent`, `Installation`, and `Tenant`, returning client/display/hostname and duration fields.
|
|
- Produces `list_blockers(..., client=None, installation=None, status="OPEN", age=None, search=None)` with summaries, priority, age, and stable pagination.
|
|
- Produces `blocker_detail(session, blocker_id)` with blocker, installation identity, related events, related jobs, and resolution audit rows.
|
|
|
|
- [ ] **Step 1: Add failing tests for dashboard reconciliation and metric semantics**
|
|
|
|
Seed a small SQLite fixture with five health states, two blockers for one product, one blocker for another product, and distinct clients. Assert that `agents_total` equals the five state sum, `safe_blocked_products == 2`, and `open_blockers == 3`.
|
|
|
|
- [ ] **Step 2: Add failing tests for attention ordering and limit**
|
|
|
|
Seed more than ten installations with mixed health, required blockers, and outdated versions. Assert that the first items are `FAILED`, then `DEGRADED`, `OFFLINE`, `UNKNOWN`, and that the result length is at most ten. Assert every item contains client, display name, hostname, problem, age, and link target.
|
|
|
|
- [ ] **Step 3: Add failing tests for agents, jobs, blockers, and detail view-models**
|
|
|
|
Assert agent rows expose percentage and separate technical/fiscal labels; job rows expose client/display/hostname and `3 min 14 s`; blocker filters reduce the result server-side; detail rows expose evidence, related job/event IDs, and audit resolution history.
|
|
|
|
- [ ] **Step 4: Implement bounded SQL/query helpers**
|
|
|
|
Use one installation join for agent/dashboard data, aggregate open blockers with `count(*)` and `count(distinct (tenant_id, product_code))` semantics compatible with SQLite/PostgreSQL, and bulk-load related job/event/audit rows for details. Keep page size capped at 100. Map blocker fields to `CFOP`, `ST`, `ICMS`, `CSOSN`, `NCM`, or `OTHER`. Never turn `safe_blocked` into technical `DEGRADED`.
|
|
|
|
- [ ] **Step 5: Run view-model tests and verify query count does not grow per row**
|
|
|
|
Run: `python -m pytest central/tests/test_operational_view_models.py central/tests/test_control_plane.py -q`
|
|
|
|
Expected: PASS; dashboard and list functions execute bounded joins without per-installation detail queries.
|
|
|
|
- [ ] **Step 6: Commit the slice**
|
|
|
|
```text
|
|
git add central/app/control_plane.py central/tests/test_control_plane.py central/tests/test_operational_view_models.py
|
|
git commit -m "feat: add operational control panel view models"
|
|
```
|
|
|
|
### Task 3: Safe Blocker Resolution Workflow
|
|
|
|
**Files:**
|
|
- Create: `central/app/blocker_resolution.py`
|
|
- Modify: `central/app/context_keys.py` only to expose the existing enum choices cleanly to the form
|
|
- Modify: `central/app/control_api.py`
|
|
- Modify: `central/app/web.py`
|
|
- Modify: `central/tests/test_control_plane.py`
|
|
- Create: `central/tests/test_blocker_resolution_panel.py`
|
|
|
|
**Interfaces:**
|
|
- `validate_resolution_input(payload) -> normalized, error` validates `key`, `value`, `scope`, `scope_key`, and `reason` through `validate_entry`, allowing only `PRODUCT`, `GROUP`, and `COMPANY` in this workflow.
|
|
- `resolve_blocker_with_context(session, blocker_id, admin, payload, request_ip=None) -> {"blocker_id": int, "job_id": int, "audit_id": int}` performs one transaction.
|
|
- Human routes are `GET /blockers/{blocker_id}` and `POST /blockers/{blocker_id}/resolve-context`; JSON API gains the same validated behavior without accepting arbitrary context text.
|
|
|
|
- [ ] **Step 1: Write failing validation, CSRF, authorization, and audit tests**
|
|
|
|
Test valid `entry_scope=INTER` and `product_role=RESALE`; reject invalid enum values, unknown keys, unsupported `NCM` workflow scope, empty reason, missing product scope key, missing CSRF, unauthenticated access, and non-admin access. Assert successful resolution creates `FiscalContextEntry`, a `REPROCESS_PRODUCTS` job, and an audit context containing before/after and job ID.
|
|
|
|
- [ ] **Step 2: Run the focused tests and verify the new route/service fails**
|
|
|
|
Run: `python -m pytest central/tests/test_blocker_resolution_panel.py -q`
|
|
|
|
Expected: FAIL because the service, detail route, and form route do not yet exist.
|
|
|
|
- [ ] **Step 3: Implement normalized validation and the transaction service**
|
|
|
|
Derive `company_key` from the installation/company context, derive product scope key from the blocker product code, read the current matching `FiscalContextEntry` as `before`, upsert the normalized entry, create a `REPROCESS_PRODUCTS` job with exactly one product code, and record a sanitized audit event with `admin_user_id`, tenant, installation, product, blocker, context key/scope/value, reason, before, after, and `reprocess_job_id`. Commit once; rollback on any validation or persistence error.
|
|
|
|
- [ ] **Step 4: Implement detail and form routes**
|
|
|
|
Render all required blocker evidence before the form, show select options for known enums, preserve CSRF, return a readable validation error on invalid input, redirect to the blocker detail after success, and remove the old `data-confirm`/hidden-context form.
|
|
|
|
- [ ] **Step 5: Run focused tests and verify security boundaries**
|
|
|
|
Run: `python -m pytest central/tests/test_blocker_resolution_panel.py central/tests/test_control_plane.py -q`.
|
|
|
|
Expected: PASS; no route accepts `AI_WRITE_ALLOWED`, `FIREBIRD_WRITES`, SQL, shell, or arbitrary context keys.
|
|
|
|
- [ ] **Step 6: Commit the slice**
|
|
|
|
```text
|
|
git add central/app/blocker_resolution.py central/app/context_keys.py central/app/control_api.py central/app/web.py central/tests/test_blocker_resolution_panel.py central/tests/test_control_plane.py
|
|
git commit -m "feat: add audited blocker context resolution"
|
|
```
|
|
|
|
### Task 4: Server-Rendered Operational Pages
|
|
|
|
**Files:**
|
|
- Modify: `central/app/web.py`
|
|
- Modify: `central/templates/base.html`
|
|
- Modify: `central/templates/dashboard.html`
|
|
- Modify: `central/templates/agents.html`
|
|
- Modify: `central/templates/agent_detail.html`
|
|
- Modify: `central/templates/blockers.html`
|
|
- Create: `central/templates/blocker_detail.html`
|
|
- Modify: `central/templates/jobs.html`
|
|
- Modify: `central/static/control-panel.css`
|
|
- Modify: `central/static/control-panel.js`
|
|
- Create: `central/tests/test_control_panel_pages.py`
|
|
|
|
**Interfaces:**
|
|
- Existing URLs remain stable: `/`, `/agents`, `/agents/{agent_id}`, `/blockers`, `/jobs`, `/login`.
|
|
- New URL: `/blockers/{blocker_id}`; POST resolution is `/blockers/{blocker_id}/resolve-context`.
|
|
- Query strings are server-side filters and remain in pagination links.
|
|
|
|
- [ ] **Step 1: Add failing page-render tests**
|
|
|
|
Log in with the test admin and assert dashboard HTML contains all five health labels, separate `Produtos bloqueados` and `Blockers abertos`, `ATENÇÃO NECESSÁRIA`, `Ver todos`, failed-job and outdated sections, and no raw ISO timestamp. Assert agents contain client/display/hostname, fiscal percentage, and separate health/status badges. Assert jobs contain client/hostname/duration. Assert blockers contain visible filter labels, summary metrics, priority, detail links, and no `window.confirm`.
|
|
|
|
- [ ] **Step 2: Run page tests and verify they fail**
|
|
|
|
Run: `python -m pytest central/tests/test_control_panel_pages.py -q`
|
|
|
|
Expected: FAIL against the current templates and routes.
|
|
|
|
- [ ] **Step 3: Implement route context and filters**
|
|
|
|
Pass the view-model results and current filter values to templates. Add agent worst-coverage sort, blocker filter parameters, job status/search parameters, event filters on agent detail, and bounded pagination links. Keep all data server-rendered.
|
|
|
|
- [ ] **Step 4: Replace templates with operational Portuguese UI**
|
|
|
|
Use accessible `<label>` elements, table `<caption>`/`<thead>` semantics, visible badges with text, empty states, detail links, technical IDs in secondary/details areas, and `title` attributes for full timestamps. Render health and fiscal badges separately.
|
|
|
|
- [ ] **Step 5: Update CSS and minimal JavaScript**
|
|
|
|
Add responsive grid breakpoints for 1366/768/tablet widths, horizontal table scrolling, visible focus styles, sufficient contrast, status styling that does not rely on color alone, and only harmless progressive enhancement. Remove the generic confirmation click handler for blocker resolution.
|
|
|
|
- [ ] **Step 6: Run page tests and commit**
|
|
|
|
Run: `python -m pytest central/tests/test_control_panel_pages.py -q`.
|
|
|
|
Expected: PASS.
|
|
|
|
```text
|
|
git add central/app/web.py central/templates central/static central/tests/test_control_panel_pages.py
|
|
git commit -m "feat: refine operational control panel pages"
|
|
```
|
|
|
|
### Task 5: Fixture Coverage, Documentation, And Regression Tests
|
|
|
|
**Files:**
|
|
- Modify: `central/scripts/generate_control_panel_fixture.py`
|
|
- Modify: `central/tests/test_control_panel_fixture.py`
|
|
- Modify: `docs/CONTROL_PANEL_V1.md`
|
|
- Create: `docs/CONTROL_PANEL_OPERATIONAL_UX_V1.md`
|
|
- Modify: `central/tests/test_control_plane.py` and focused test modules as needed
|
|
|
|
**Interfaces:**
|
|
- The existing fixture command remains explicit and non-production-only.
|
|
- Fixture includes all five health states, current/outdated/unknown versions, failed jobs, safe-blocked products, and multiple blockers for at least one product.
|
|
|
|
- [ ] **Step 1: Extend fixture tests before changing fixture data**
|
|
|
|
Assert the fixture has multiple blockers sharing a tenant/product, failed jobs, all health/version states, and enough client/server identity fields for every operational table.
|
|
|
|
- [ ] **Step 2: Update fixture deterministically**
|
|
|
|
Keep the existing 100-agent shape and return contract. Add a second blocker for selected products, varied blocker types/fields and reasons, explicit finished/started timestamps for jobs, and events carrying job/product/blocker references without secrets.
|
|
|
|
- [ ] **Step 3: Add documentation tests/contract assertions**
|
|
|
|
Check the documentation names health-vs-fiscal semantics, safe-blocked-vs-open-blocker definitions, attention ordering/limit, timezone setting, resolution audit flow, and production password policy.
|
|
|
|
- [ ] **Step 4: Update both documentation files**
|
|
|
|
Document Portuguese operational labels, URL/filter examples, the resolution workflow, supported context enums/scopes, no-write boundaries, fixture command, and `UI_TIMEZONE`/`ALLOW_WEAK_DEV_PASSWORD` configuration.
|
|
|
|
- [ ] **Step 5: Run all central tests and commit**
|
|
|
|
Run: `python -m pytest central/tests -q`.
|
|
|
|
Expected: PASS with only existing non-failing warnings.
|
|
|
|
```text
|
|
git add central/scripts/generate_control_panel_fixture.py central/tests docs/CONTROL_PANEL_V1.md docs/CONTROL_PANEL_OPERATIONAL_UX_V1.md
|
|
git commit -m "test: cover operational panel fixture and contracts"
|
|
```
|
|
|
|
### Task 6: Runtime, Performance, And Full Verification
|
|
|
|
**Files:**
|
|
- Modify only files required by failing verification; do not add unrelated refactors.
|
|
- Create no runtime artifacts containing credentials or session values.
|
|
|
|
- [ ] **Step 1: Run the complete offline regression**
|
|
|
|
Run exactly:
|
|
|
|
```text
|
|
python -m pytest tests -q
|
|
python -m pytest central/tests -q
|
|
python -m pytest agent/tests -q
|
|
```
|
|
|
|
Expected: all suites pass; record counts and non-failing warnings.
|
|
|
|
- [ ] **Step 2: Rebuild and start only the local Hub stack**
|
|
|
|
Run: `docker compose up -d --build fiscal-hub`.
|
|
|
|
Verify PostgreSQL is healthy and `GET http://localhost:8000/health` returns `200`. Do not deploy or publish additional ports.
|
|
|
|
- [ ] **Step 3: Run authenticated route smoke checks**
|
|
|
|
Using the existing local admin without printing its password or cookies, request `/`, `/agents`, `/blockers`, `/jobs`, one agent detail, one blocker detail, and submit one valid resolution form with CSRF in a disposable/local fixture context. Verify redirects, `200` responses, no raw ISO timestamps in primary HTML, and no secret values in response/log output.
|
|
|
|
- [ ] **Step 4: Run bounded performance smoke**
|
|
|
|
Against the 100-agent fixture, time dashboard, agents, blockers, jobs, and agent detail requests with a monotonic clock. Report the maximum route time in milliseconds and fail the check if any route exceeds 1000 ms locally or if a route produces an unbounded query loop.
|
|
|
|
- [ ] **Step 5: Verify safety flags and repository state**
|
|
|
|
Check `AI_WRITE_ALLOWED=false`, `FIREBIRD_AI_WRITES=0`, no remote Firebird write path, no deployment command, `git diff --check`, `git status --short`, and recent commit list. Ensure no password, token, cookie, or secret appears in tracked files or generated reports.
|
|
|
|
- [ ] **Step 6: Produce the consolidated mission report**
|
|
|
|
Report `STATUS`, all requested UX readiness flags, measured `MAX_ROUTE_TIME_MS`, test counts, PostgreSQL/Hub health, safety flags, commits, Git status, and a verdict that distinguishes completed behavior from any residual warning.
|