gustavosantos/GTI_GERENCIADOR_FISCAL
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
7e83d17533c7166d6b58207c98d061f38e4e732b
GTI_GERENCIADOR_FISCAL/docs/superpowers/plans/2026-09-22-gti-fiscal-control-panel-v1.md
T

16 KiB
Raw Blame History

GTI Fiscal Control Panel V1 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a tested FastAPI/Jinja2 control panel that identifies every Agent installation, exposes deterministic health and coverage, centralizes redacted events, and supports safe authenticated operations.

Architecture: Keep the existing Hub as the only application boundary. Add focused SQLAlchemy models and query/service modules for installations, snapshots, events, human sessions, and audits. Keep Agent Bearer authentication separate from human cookie sessions, and render the operator UI with Jinja2, CSS, and small JavaScript.

Tech Stack: FastAPI, SQLAlchemy 2, Alembic, PostgreSQL, Jinja2, standard-library PBKDF2-HMAC password hashing, HttpOnly cookie sessions, server-rendered HTML, CSS, vanilla JavaScript.

Global Constraints

  • AI_WRITE_ALLOWED=false and FIREBIRD_AI_WRITES=0 remain unchanged.
  • No fiscal decision logic changes, production deployment, real-client installation, raw log upload, SQL execution, shell execution, or remote Windows restart.
  • Agent authentication remains Bearer/token based; human authentication uses a separate expiring cookie session.
  • installation_id is a persistent UUID and IP is never a primary identity.
  • SAFE_BLOCKED does not independently make an Agent degraded or failed.
  • All human mutations require authentication, CSRF validation, explicit confirmation, and an audit event.
  • Dashboard queries use snapshots and bounded aggregates; list endpoints are paginated with bounded page sizes.
  • Existing tests, central/tests, and agent/tests must remain passing.

File Map

Foundation

  • Create central/alembic/versions/0017_control_plane_foundation.py: installation, snapshot, event, admin/session/audit tables and indexes.
  • Modify central/app/models.py: focused models for the new tables.
  • Modify central/app/config.py: heartbeat/offline, retention, session, cookie, and version settings.
  • Create central/app/installation_identity.py: UUID generation, masked CNPJ/path, metadata allowlist, health/version functions.
  • Create central/app/events.py: event type validation, redaction, size bounds, ingestion and retention.
  • Create central/app/control_plane.py: snapshot aggregation, summary queries, filters, and pagination services.
  • Create central/app/human_auth.py: password hashing, admin lookup, session and CSRF helpers.
  • Create central/app/audit.py: immutable human action audit helper.
  • Modify central/app/api.py: enriched Agent registration/heartbeat and Agent event/snapshot ingestion routes; preserve existing routes.
  • Modify central/app/main.py: mount foundation/control-plane routes and templates/static files.
  • Modify central/app/auth.py: registration lookup by installation identity without weakening Bearer auth.
  • Modify agent/src/gti_fiscal_agent/state.py: persist installation UUID and local registration metadata.
  • Modify agent/src/gti_fiscal_agent/client.py: registration, enriched heartbeat, event sending methods.
  • Modify agent/scripts/run_agent_service.py: load stable installation identity and send structured lifecycle/heartbeat metadata.
  • Create central/scripts/create_admin.py: interactive secure admin creation CLI.
  • Create central/scripts/maintain_events.py: retention maintenance command.

APIs And Web

  • Create central/app/control_api.py: human-authenticated JSON dashboard, agents, blockers, jobs, and events endpoints.
  • Create central/app/web.py: login/logout, dashboard/list/detail pages, safe action form handlers, CSRF and confirmation checks.
  • Create central/templates/base.html, login.html, dashboard.html, agents.html, agent_detail.html, blockers.html, jobs.html, events.html.
  • Create central/static/control-panel.css, central/static/control-panel.js.
  • Create central/tests/test_control_plane.py, test_human_auth.py, test_control_api.py, test_web_panel.py, test_event_security.py.
  • Create central/scripts/generate_control_panel_fixture.py: explicit non-production 100-Agent fixture generator.
  • Create central/tests/test_control_panel_fixture.py: fixture distribution and bounded query checks.
  • Create docs/CONTROL_PANEL_V1.md: architecture, auth, identity, health, events, retention, actions, endpoints, screens, operations.
  • Modify docs/AI_PROJECT_STATE.md: record panel readiness and security invariants.

Tasks

Task 1: Add Persistent Control-Plane Schema

Files: migration and model files above.

Interfaces: Models expose Installation, AgentSnapshot, AgentEvent, AdminUser, WebSession, and AuditEvent; later services use installation_id, indexed event timestamps, snapshot counts, and session hashes.

  • Write model tests asserting UUID uniqueness, one installation per Agent, snapshot upsert shape, event indexes, admin/session uniqueness, and audit append-only fields.
  • Add the models with nullable metadata for backward-compatible existing Agents. Store installation_id as a 36-character UUID string with a unique index; store secrets only as hashes.
  • Add migration 0017_control_plane_foundation with foreign keys to tenants, agents, jobs, and blocked_products where applicable. Add indexes on tenant/installation, health/last_seen, snapshot update time, event installation/time/severity/type, job agent/status/time, and blocker tenant/type/resolution.
  • Run python -m pytest central/tests/test_control_plane.py -q and apply migration only to the configured development PostgreSQL database when available.
  • Commit feat: add control plane foundation schema.

Task 2: Implement Installation Identity And Agent Metadata

Files: identity service, existing Agent auth/API, Agent state/client/service, tests.

Interfaces: ensure_installation_id(state_path: str) -> str, sanitize_registration(payload: dict) -> dict, mask_cnpj(value: str | None) -> str | None, sanitize_database_path(value: str | None) -> str | None, and agent_registration(payload, bearer) -> dict.

  • Write failing tests proving the same local state path returns the same UUID, a new explicit installation creates a new UUID, hostname/IP changes do not change identity, and registration never stores password/token/API-key/connection-string values.
  • Add an atomic local installation_id record to AgentState and include it in registration/heartbeat payloads. Keep the existing agent_id field separate.
  • Extend registration/heartbeat metadata with display name, customer code, notes, OS/version, machine fingerprint, masked company CNPJ, UF/CRT, Firebird version/ODS, sanitized database identifier, capabilities, fiscal engine, datasets, started time, and last job data.
  • Upsert the Hub installation record by (agent_id, installation_id) and reject conflicting installation IDs rather than silently reassigning an existing installation.
  • Run python -m pytest central/tests/test_control_plane.py agent/tests/test_state.py agent/tests/test_run_agent_service.py -q.
  • Commit feat: persist agent installation identity and metadata.

Task 3: Implement Health, Version Drift, And Snapshots

Files: central/app/installation_identity.py, central/app/control_plane.py, config, heartbeat/job completion routes, tests.

Interfaces: compute_agent_health(snapshot, now) -> str, compute_version_drift(agent_version, server_version) -> str, upsert_agent_snapshot(session, installation, payload) -> AgentSnapshot, dashboard_summary(session, filters) -> dict, and paginate(query, page, page_size) -> dict.

  • Write tests for UNKNOWN, recent healthy ONLINE, recent technical failure DEGRADED, persistent technical failure FAILED, stale OFFLINE, and safe-blocked-only ONLINE.
  • Implement configurable AGENT_HEARTBEAT_INTERVAL_SECONDS, AGENT_OFFLINE_AFTER_SECONDS with a three-interval default, CURRENT_SERVER_VERSION, and AGENT_EVENT_RETENTION_DAYS=90.
  • Persist product counts, coverage, failed products, blocker aggregates, last/last-successful job, heartbeat, health, and version drift in one upserted snapshot.
  • Add dashboard aggregate queries that only read snapshot rows plus bounded grouped counts. Normalize datetimes as UTC for SQLite and PostgreSQL tests.
  • Run health/snapshot tests and existing job tests; verify SAFE_BLOCKED does not degrade health.
  • Commit feat: add deterministic agent health and snapshots.

Task 4: Add Structured Events, Redaction, Retention, And Audit

Files: central/app/events.py, central/app/audit.py, models/migration if needed, API routes, maintenance CLI, tests.

Interfaces: sanitize_event(message, context) -> tuple[str, dict], record_event(session, *, installation_id, severity, event_type, message, context, job_id=None, product_code=None, blocker_id=None) -> AgentEvent, purge_expired_events(session, now, retention_days) -> int, and record_audit(session, actor, action, target, context) -> AuditEvent.

  • Write tests covering case-insensitive redaction of passwords, tokens, API keys, connection strings and secret env names in both messages and nested JSON; assert event message/context maximums.
  • Implement allowlisted event types and severities from the mission. Store structured context only, truncate safely, and never persist raw log files.
  • Record Agent lifecycle, heartbeat, preflight, job, blocker, dataset/schema, connection, retry, version, and write-audit events at the relevant existing route boundaries.
  • Implement retention deletion for non-audit events older than AGENT_EVENT_RETENTION_DAYS; preserve audit events, blocker history, write audit, and critical events.
  • Add python -m central.scripts.maintain_events --dry-run and a write mode that only deletes eligible events.
  • Run event security and retention tests, then commit feat: add redacted operational events and retention.

Task 5: Implement Human Admin Authentication And Session Security

Files: central/app/human_auth.py, central/app/web.py, config/main, CLI, tests.

Interfaces: hash_password(password) -> str, verify_password(password, encoded_hash) -> bool, create_admin(session, username, password) -> AdminUser, create_web_session(session, admin, user_agent, ip) -> tuple[str, str], require_human(request) -> AdminUser, and require_csrf(request, submitted_token) -> None.

  • Write tests for PBKDF2 password storage, wrong-password rejection, inactive admin rejection, session expiry, cookie flags, session rotation, logout invalidation, and CSRF mismatch/replay rejection.
  • Add PBKDF2-HMAC-SHA256 hashes with random salts and constant-time verification using only the standard library. Store only session token hashes and CSRF token hashes server-side.
  • Add gti_admin_session HttpOnly cookie with Secure according to production config, SameSite=Lax, bounded max age, and no Agent token reuse.
  • Add python -m central.scripts.create_admin using getpass, refusing empty/mismatched passwords and duplicate usernames.
  • Run auth tests and commit feat: add separate human admin sessions.

Task 6: Add Control-Plane APIs And Safe Actions

Files: central/app/control_api.py, API integration, audit/events, tests.

Interfaces:

GET  /v1/dashboard/summary
GET  /v1/agents
GET  /v1/agents/{agent_id}
GET  /v1/agents/{agent_id}/health
GET  /v1/agents/{agent_id}/coverage
GET  /v1/agents/{agent_id}/blockers
GET  /v1/agents/{agent_id}/jobs
GET  /v1/agents/{agent_id}/events
GET  /v1/blockers/global
GET  /v1/jobs/global
POST /v1/agents/{agent_id}/actions/reprocess
POST /v1/blockers/{blocker_id}/resolve-human
  • Write tests for authentication, client/health/version/last-seen/blocker/severity/date filters, bounded pagination, empty pages, and cross-tenant access denial.
  • Implement query services with page, page_size, total, items, max page size 100, explicit sort allowlists, and no relationship N+1 queries.
  • Implement safe actions only: create idempotent reprocess jobs, resolve only with explicit resolution context, and record actor/action/target/context audit events.
  • Reject requests that attempt Firebird writes, AI flags, credentials, database paths, CRT changes, SQL, shell, restart, or arbitrary action names.
  • Run API tests and commit feat: expose authenticated control plane APIs.

Task 7: Build Server-Rendered Operational UI

Files: templates, CSS/JS, central/app/web.py, main, tests.

Interfaces: browser routes /login, /logout, /, /agents, /agents/{agent_id}, /blockers, /jobs, and /events; all mutations use POST forms with hidden CSRF and a visible confirmation field.

  • Write template tests asserting dashboard cards, client/display-name/hostname identity, health indicator, blocker/job/event pagination, technical IDs only in detail views, and no secret values in rendered HTML.
  • Implement dashboard cards for Agent health, product coverage, technical failures, recent failed jobs, stale heartbeats, version drift, and top blocker types.
  • Implement responsive Agents/Servers table with search/sort/filter/page controls and links to detail tabs: overview, fiscal, blockers, jobs, events, capabilities, technical info.
  • Implement global blockers/jobs/event pages and per-installation timelines with bounded pagination and filters.
  • Add plain CSS responsive layout, recognizable health badges, compact tables, confirmation dialogs, and minimal JavaScript for filters/confirmations only.
  • Run python -m pytest central/tests/test_web_panel.py -q and commit feat: add server-rendered control panel.

Task 8: Add Fixture Generator And Performance Checks

Files: fixture script, tests, optional fixture helpers.

Interfaces: generate_fixture(session, agent_count=100, seed=...) -> dict; must be explicit and never run during app startup or migration.

  • Write tests asserting exactly 100 installations with distributed ONLINE, DEGRADED, OFFLINE, and FAILED states, blockers, jobs, and events.
  • Implement deterministic fixture generation behind an explicit --database-url and --confirm-non-production guard; refuse production APP_ENV and refuse empty confirmation.
  • Add query-count/performance checks that dashboard uses snapshot rows and event/job pages issue bounded queries for 100 Agents and large event counts.
  • Run fixture tests without seeding the configured production database and commit test: add control panel fixture and query checks.

Task 9: Document, Run Full Verification, And Consolidate

Files: docs/CONTROL_PANEL_V1.md, docs/AI_PROJECT_STATE.md, final tests.

  • Document architecture, identity lifecycle, metadata allowlist, health thresholds, event taxonomy/redaction, retention command, auth/session/CSRF, safe actions, endpoints, screens, and fixture usage.
  • Update project state with control-plane readiness and explicit AI_WRITE_ALLOWED=false, FIREBIRD_AI_WRITES=0, DEPLOY_PERFORMED=false.
  • Run python -m pytest tests -q.
  • Run python -m pytest central/tests -q.
  • Run python -m pytest agent/tests -q.
  • Run git diff --check, inspect git status, and verify no secrets or generated fixture database are tracked.
  • Commit docs: document control panel v1 operations.

Verification Matrix

Requirement Plan task Evidence
Persistent installation identity and metadata 1-2 identity tests, registration tests, migration
Deterministic health/offline/version drift 3 health/snapshot tests
Coverage and blocker summaries 3, 6-7 snapshot/API/template tests
Structured redacted events 4 redaction, size, ingestion tests
Retention and audit preservation 4 maintenance tests
Human auth, cookie session, CSRF 5 auth/session tests
Paginated filtered APIs and safe actions 6 API and audit tests
Dashboard and operational screens 7 template and route tests
100-Agent simulation/performance 8 fixture/query checks
Documentation and regression 9 docs and three suites
Reference in New Issue View Git Blame Copy Permalink