# GTI Fiscal — Pilot Deployment Guide Living document for operating a real pilot: one Hub, one or more client Agent installations. See `docs/OPERATIONS.md` for the day-to-day command reference (creating jobs, checking status); this document is about getting a new environment stood up and kept alive. ## RC1 (1.1.0) — supported path ``` Hub central (Docker: Hub + PostgreSQL, deploy/hub) ^ HTTPS (TLS terminated by a reverse proxy) | Agent Windows (service "GTIFiscalAgent", deploy/agent-windows) | local read-only connection SGV Firebird (the client's C000025 / C000004) ``` The Hub **never** accesses Firebird, never stores Firebird credentials and never holds the client's database path. Only the Agent, on the client's Windows host, opens Firebird. RC1 pilots run `FISCAL_MODE=DRY_RUN` + `FIREBIRD_WRITES=0`: the Agent is technically unable to write (no write connection is ever opened, the executor is never called). ### Hub (GTI infrastructure) 1. `cd deploy/hub && cp .env.example .env` — set `POSTGRES_PASSWORD` (`openssl rand -hex 24`), `BASE_URL` (public HTTPS URL). Leave `HUB_BIND=127.0.0.1` and publish it through the reverse proxy. 2. `docker compose -f docker-compose.prod.yml --env-file .env up -d --build` — PostgreSQL (internal network only, persistent volume, healthcheck), Hub (migrations at start, healthcheck, `restart: unless-stopped`, json-file logs 10 MB x 5). 3. Seed the validated datasets once: on the development machine `sh export_datasets_seed.sh > gti_datasets_seed.dump`; on the Hub host `sh seed_datasets.sh gti_datasets_seed.dump` (refuses if `datasets` is not empty; reference tables only, never tenants/agents/jobs). 4. Register the Agent: `docker compose -f docker-compose.prod.yml exec fiscal-hub python central/scripts/register_agent.py --company "" --name ""` — the token is printed once; store it only in the Agent's local `.env`. 5. Health: `GET /health` (Postgres included), `GET /version` (release/engine/decision versions). `APP_ENV=production` refuses to start when `AGENT_DEV_TOKEN` is set. 6. Backup/restore: `sh backup.sh` (pg_dump -Fc into `BACKUP_DIR`, keeps 14) and `sh restore.sh ` (asks for the word RESTORE, stops the Hub, restores, restarts; migrations rerun at start). ### Agent (client host, PowerShell as Administrator, inside the extracted `GTIFiscalAgent-` folder) Build the package on the release machine: `python scripts/build_agent_package.py` -> `dist/GTIFiscalAgent-.zip` (the version is `release/VERSION`) (+ `.sha256`; secret-scanned, no `.env`). | step | command | |---|---| | install | `.\gti-agent.ps1 install` (venv + dependencies + service registration; does not start) | | configure | `.\gti-agent.ps1 configure` (prompts; secrets never echoed) or `configure -FromTemplate .\pilot-construcao.env.example` | | preflight | `.\gti-agent.ps1 preflight` -> `PREFLIGHT=PASS|FAIL` + blockers | | start / stop / restart | `.\gti-agent.ps1 start` (refuses when preflight fails) | | status | `.\gti-agent.ps1 status` | | uninstall | `.\gti-agent.ps1 uninstall [-Purge]` | ### Agent configuration (per installation, `.env` in the install folder) `GTI_HUB_URL`, `AGENT_ID`, `AGENT_TOKEN`; `COMPANY_CODE` / `COMPANY_CNPJ` (company selection), `COMPANY_CRT_SOURCE` = `AUTO` | `EMISSOR` | `C000004`; `COMPANY_UF` only as an explicit override (it never replaces the resolver: it must agree with what the database proves); `FIREBIRD_DATABASE_PATH`, `FB_USER` (read-only account), `FB_PASSWORD`; `FISCAL_MODE=DRY_RUN`, `FIREBIRD_WRITES=0`; `POLL_INTERVAL_SECONDS`, `AGENT_VERSION`. `AUTO` resolves the CRT only when EMISSOR and C000004 agree; a conflict makes the preflight FAIL until `EMISSOR` or `C000004` is chosen explicitly. The first row of C000004 is never used to pick the company. ### Preflight (`agent/scripts/preflight.py`, also enforced at service start) HUB_REACHABLE, AUTH, HUB_VERSION_COMPATIBLE (major.minor), FIREBIRD_REACHABLE, FIREBIRD_VERSION_ODS, C000004, C000025, CAPABILITIES (required: NCM, IPI, CEST), COMPANY_CONTEXT (UF, CRT), SCHEMA_TYPE_GUARDS, SQLITE, LOG_DIRECTORY, FISCAL_MODE, FIREBIRD_WRITES (`DRY_RUN` + `FIREBIRD_WRITES=1` is a misconfiguration -> FAIL). ### Pilot profiles `deploy/agent-windows/pilot-construcao.env.example` (first pilot: company 000001, UF BA, CRT 1, `COMPANY_CRT_SOURCE=AUTO`, DRY_RUN). Planned, **not installed**: `profiles/mmmodas|rincao|uniao.env.example` (`COMPANY_CRT_SOURCE=EMISSOR` required: EMISSOR and C000004 disagree on the CRT). Never reuse the path of a local database copy as the client's real path. ### Zero-write proof `python scripts/prove_zero_write.py snapshot before.json` ... run the pilot ... `snapshot after.json` ... `compare before.json after.json` (database DML counters, SHA-256 of every row of C000025/C000004, row counts). --- The sections below are the previous manual instructions (still accurate for development); the RC1 scripts above are the supported way to deploy. ## Architecture ``` ┌────────────────────────────┐ │ Hub server (GTI/Proxmox) │ │ Docker Compose: │ │ - postgres (data volume) │ │ - fiscal-hub (FastAPI) │ └─────────────┬────────────────┘ │ HTTPS (reverse proxy terminates TLS) ┌────────────────────────┼────────────────────────┐ │ │ │ ┌─────────▼─────────┐ ┌─────────▼─────────┐ ┌─────────▼─────────┐ │ Client A — Agent │ │ Client B — Agent │ │ Client C — Agent │ │ Windows Service │ │ Windows Service │ │ Windows Service │ │ + local Firebird │ │ + local Firebird │ │ + local Firebird │ │ + local SQLite audit│ │ + local SQLite audit│ │ + local SQLite audit│ └────────────────────┘ └────────────────────┘ └────────────────────┘ ``` The Hub never has Firebird credentials and never opens a Firebird connection. Each Agent has its own token and only ever reads/writes the one Firebird database it was configured for. ## Hub server requirements - Docker + Docker Compose v2. - Enough disk for the Postgres volume (starts small; grows with `classification_cache`/`jobs` history — no large binary data). - One inbound port for the Hub API (default `8000`, or whatever your reverse proxy forwards to it). Postgres itself does **not** need a host port — `docker-compose.yml` intentionally does not publish `5432`; only containers on the compose network reach it. ## Deploying the Hub ```bash git clone && cd Projeto cp .env.example .env # Edit .env: set a real POSTGRES_PASSWORD, leave AGENT_DEV_TOKEN empty, # set APP_ENV=production, BASE_URL to the public HTTPS URL. docker compose up -d postgres fiscal-hub curl http://localhost:8000/health # {"status": "ok"} ``` `fiscal-hub`'s container command runs `alembic upgrade head` before starting `uvicorn`, so schema migrations apply automatically on every deploy/restart. Both services have `restart: unless-stopped` and Docker healthchecks (`pg_isready` for Postgres, an HTTP GET to `/health` for the Hub). **Never** put `AGENT_DEV_TOKEN` in a production `.env` — it auto-bootstraps an agent from a fixed shared token on every request, fine for local dev, wrong for a pilot with real client secrets. Leave it empty; register real Agents with `central/scripts/register_agent.py` (see below). ### Persistence - Postgres data lives in the named volume `postgres_data`. Back it up with `docker exec pg_dump -U gti gti_fiscal > backup.sql` (or `pg_dumpall` for the whole cluster) on whatever schedule your operations policy requires. - Always run `alembic upgrade head` (the container does this automatically) rather than hand-editing the schema. An **empty** database is initialized by that same command — never with `alembic stamp` or `create_all`. `0001` is an explicit, frozen baseline (it used to call `Base.metadata.create_all()`, which made `0002` collide on `datasets.raw_sha256` on any fresh database); `central/tests/test_migration_baseline.py` keeps that from coming back. - `0019` repairs `agents.created_at/updated_at` and `jobs.created_at`, whose defaults `0006` had frozen at the instant it ran (rows created before `0019` keep that frozen value — their real creation time is not recoverable). - `0020` enforces NOT NULL on 20 timestamp columns that always had `DEFAULT now()`. It aborts, changing nothing, if any NULL exists — clean those rows explicitly first. `0021` adds the SSO federated identity (see "Human access"); `0022` adds Agent releases + secure enrollment (see "Agent distribution"). Current head: `0022_agent_releases_enrollment` (also recorded in `release/RELEASE_MANIFEST.json`). ### HTTPS / reverse proxy The Hub itself speaks plain HTTP. For a pilot reachable outside the Hub's own host, put it behind a reverse proxy (nginx, Caddy, Traefik — whatever the GTI/Proxmox environment already standardizes on) that terminates TLS and forwards to `fiscal-hub:8000`. Nothing in the app assumes a particular proxy; `BASE_URL` in `.env` is informational only (not currently used to generate absolute links, but keep it accurate for anyone reading the config). **GTI production (2026-09-23):** TLS terminates in the GTI Nginx Proxy Manager (10.0.10.100), which forwards `http://:8000`. On the Hub host: - `.env`: `HUB_BIND=` (not `0.0.0.0`), `FORWARDED_ALLOW_IPS=` (real client IP and `https` scheme from `X-Forwarded-*`), `BASE_URL=https://…`, and `WEB_COOKIE_SECURE=1` once HTTPS works. Redirects are relative, so the internal address never leaks. - Docker-published ports bypass UFW: install `restrict_hub_port.sh` + `gti-fiscal-hub-port.service` (instructions in the unit file) so only the NPM reaches the port, re-applied on every Docker start. ## Human access: SSO from GTI Gerenciador Collaborators never type a second password. GTI Gerenciador (the panel) is the identity authority; the Hub keeps its own sessions. 1. "Fiscal" in the panel sidebar -> panel `/sso/fiscal` -> Hub `GET /login`, which sets a one-time state cookie (`gti_sso_state`, HttpOnly, Secure, SameSite=None, 5 min) and redirects to `SSO_START_URL?state=...&next=...`. 2. The panel requires its own login (normal login page if needed), checks the employee is active and auto-POSTs a JWT to `/auth/sso/callback`: Ed25519 (`EdDSA` only), `iss=gti-gerenciador`, `aud=gti-fiscal`, `sub=funcionario:`, 60 s, unique `jti`, `nonce`=state. Never in a URL. 3. The Hub verifies signature/iss/aud/exp/nbf/lifetime, the nonce against the state cookie (login-CSRF protection), records the `jti` (`sso_used_assertions`, replay = rejected), provisions or refreshes the collaborator in `admin_users` (`auth_source=GTI_GERENCIADOR`, no password, enforced by `ck_admin_users_auth_source`) and opens a session of `FEDERATED_SESSION_SECONDS` (default 1 h; afterwards the same flow signs the user in again silently, which also picks up a deactivation in the panel). Keys: a dedicated Ed25519 pair, used for nothing else (never reuse SESSION_SECRET, admin, Agent or DB secrets). The **private** key (raw 32-byte seed, base64url) goes only to the panel's `.env` (`SSO_FISCAL_CHAVE_PRIVADA`); the **public** key only to this Hub's `.env` (`SSO_GERENCIADOR_CHAVE_PUBLICA`). Rotating = new pair in both, restart both. Logout ("Sair do Fiscal") ends only the Hub session; coming back through the panel signs in again without a password. There is no distributed single logout. Audit (`audit_events`): `SSO_LOGIN_SUCCESS`, `SSO_LOGIN_REJECTED` (with a stable reason: `EXPIRED`, `INVALID_SIGNATURE`, `WRONG_AUDIENCE`, `WRONG_ISSUER`, `REPLAY`, `STATE_MISMATCH`, `DISABLED_IN_FISCAL`, ...), `LOGOUT`, `LOGIN_LOCAL_SUCCESS/FAILURE` — with the real client IP, never the token. **Break-glass:** the local admin (`create_admin.py`) still exists for emergencies at `/login/local` (not linked from the UI; every use is audited). Disable a collaborator only in the Hub: `admin_users.is_active = false`. ## Registering an Agent (per client installation) Run once per Agent installation, from anywhere with access to Postgres — easiest is inside the running Hub container: ```bash docker exec python central/scripts/register_agent.py \ --company "Client Company Name" --name "loja-01" ``` This prints `AGENT_ID`, a freshly generated `AGENT_TOKEN` (shown once — copy it immediately), and `COMPANY_ID`. Only the token's SHA-256 hash is ever stored in Postgres (same `token_hash` mechanism `/v1/classify` already used — no new auth system). Each Agent gets its own token; one Agent's token never authenticates another Agent's jobs (`agent_id` is checked on every Job operation). ## Agent distribution: install from the panel (secure enrollment) An alternative to `register_agent.py` for a real pilot: the panel drives the whole installation, and the technician on the client's Windows host never gets a permanent secret by hand. **One-time: publish a release.** The Agent's universal package (one zip per version — never one per client) is built and published by script, never uploaded through a browser: ```bash python scripts/build_agent_package.py # -> dist/GTIFiscalAgent-.zip docker exec python central/scripts/publish_agent_release.py \ --zip dist/GTIFiscalAgent-.zip --notes "..." ``` Publishing the same version again with different bytes is refused (bump the version instead); publishing identical bytes again is a no-op. The zip lives on the Hub's local `agent_releases` volume (`AGENT_RELEASE_STORAGE_PATH`, default `/app/data/releases`), never in PostgreSQL — only its metadata (`agent_releases`: version, sha256, size, status DRAFT/PUBLISHED/REVOKED) is a row. `Agents / Servidores -> Releases` lists and can revoke a release. **Per installation:** `Agents / Servidores -> + Nova instalação` — pick the client, a display name, an optional expected hostname and the release, "Criar instalação". That installation's page then offers: - **Baixar Agent agora** — an authenticated, immediate download (the operator is already logged in). - **Gerar link temporário** — a one-time HTTPS URL (`/dl/agent/`), shown exactly once, TTL `AGENT_DOWNLOAD_LINK_TTL_SECONDS` (default 30 min), single use (`AGENT_DOWNLOAD_MAX_USES_DEFAULT`, default 1). Hand it to a technician or open it directly on the client's server — no login required to use it. Either path streams a bundle = the universal package + a freshly minted `enrollment.json` (installation id, a one-time enrollment token, the Hub URL) written at download time. Only that enrollment token's SHA-256 hash is ever stored (`agent_enrollments.enrollment_token_hash`); its plaintext exists only inside that one HTTP response — never on disk on the Hub, never logged. **On the client Windows host**, elevated PowerShell, inside the extracted package: ```powershell .\gti-agent.ps1 install-auto ``` This checks every file in the bundle against `MANIFEST.json`'s SHA-256, refuses a non-HTTPS Hub URL, calls `POST /v1/agents/enroll` with the enrollment token (deleting `enrollment.json` immediately after, success or failure), writes `.env` with the newly issued **permanent** `AGENT_TOKEN` (ACL-restricted to Administrators/SYSTEM, same as `configure`), and registers the Windows Service — **without starting it**: Firebird is still unconfigured on purpose (this script never guesses a client's database path or credentials, same rule as the manual path). Edit `FIREBIRD_DATABASE_PATH` / `FB_USER` / `FB_PASSWORD` / `COMPANY_CODE` in `.env`, then `.\gti-agent.ps1 preflight ; .\gti-agent.ps1 start`. Running `install-auto` again after a successful enrollment is a no-op (idempotent). The Hub rejects: an expired or already-used download link, a revoked installation or release, a wrong/tampered/replayed enrollment token, and a second enrollment attempt for an already-enrolled installation. A hostname that doesn't match what the operator expected is recorded (`hostname_mismatch`) and audited, but does not block enrollment. Both public routes (`GET /dl/agent/{token}`, `POST /v1/agents/enroll`) are rate-limited per source IP. Audit actions: `INSTALLATION_CREATED`, `DOWNLOAD_LINK_CREATED`, `DOWNLOAD_LINK_REJECTED`, `PACKAGE_DOWNLOADED`, `ENROLLMENT_SUCCEEDED`, `ENROLLMENT_REJECTED`, `AGENT_CREDENTIAL_ISSUED`, `INSTALLATION_REVOKED`, `AGENT_RELEASE_REVOKED` — the real IP, never a token. Revoking a pending or enrolled installation (`Revogar` on its detail page) invalidates any outstanding link/enrollment token immediately and, if an Agent was already provisioned, disables it (`agents.enabled = false`) — its existing sessions/jobs stop authenticating on the next request. ## Configuring an Agent installation On the client machine, copy `agent/.env.example` to `agent/.env` and fill in the values from registration plus that machine's Firebird details: ``` GTI_HUB_URL=https://hub.example.com AGENT_ID= AGENT_TOKEN= COMPANY_ID= COMPANY_UF=BA FIREBIRD_DATABASE_PATH=C:\ERP\dados\PRODUCAO.FDB FB_USER=... FB_PASSWORD=... FIREBIRD_WRITES=0 POLL_INTERVAL_SECONDS=10 AGENT_VERSION=0.1.0 ``` **`FIREBIRD_WRITES` must default to `0`.** See "Write safety" below before ever setting it to `1`. ## Installing the Agent as a Windows Service Requires `pywin32` (`pip install pywin32`, or `pip install -e ".[windows-service]"` from `agent/`). ```powershell # From an elevated (Administrator) terminal, in agent/service/: python gti_agent_windows_service.py --startup=auto install python gti_agent_windows_service.py start # One-time, so Windows restarts it automatically after a crash: sc.exe failure "GTIFiscalAgent" reset= 86400 actions= restart/60000/restart/60000/restart/60000 ``` `gti_agent_windows_service.py` is a thin wrapper: it does not reimplement polling, heartbeat, or the pipeline — it calls `run_agent_service.run()` directly, stopping it via the Service Control Manager's own stop signal instead of `SIGINT`/`SIGTERM` (which don't apply to services running with no console). See the module's docstring for the exact commands. ### Install / configure / start / status / stop / restart / uninstall ```powershell # Install (registers with SCM, Startup=Automatic) python gti_agent_windows_service.py --startup=auto install # Configure: edit agent/.env, no service restart needed until you `restart` notepad agent\.env # Start python gti_agent_windows_service.py start # Status sc.exe query GTIFiscalAgent # Stop python gti_agent_windows_service.py stop # Restart (picks up any agent/.env changes) python gti_agent_windows_service.py restart # Uninstall python gti_agent_windows_service.py stop python gti_agent_windows_service.py remove ``` ### Updating the Agent's version 1. Stop the service: `python gti_agent_windows_service.py stop` 2. Replace the `agent/` code with the new version (git pull, or copy files). 3. Bump `AGENT_VERSION` in `agent/.env` if you track it per-release. 4. Start the service again: `python gti_agent_windows_service.py start`. 5. Confirm the new version shows up via `GET /v1/agents/me` on the Hub. There is no auto-update mechanism yet (deliberately out of scope for this pilot) — updates are a manual stop/replace/start. ### Running without installing as a service (dev/testing) ```bash cd agent python scripts/run_agent_service.py # runs forever, Ctrl+C to stop python scripts/run_agent_service.py --max-iterations 5 # bounded, for scripting/tests ``` ## Creating and checking work See `docs/OPERATIONS.md` for the full `curl` examples (`POST /v1/jobs` with `kind: REPROCESS_ALL` or `REPROCESS_PRODUCTS`, `GET /v1/jobs/{id}` for status). Every Job is scoped to the Agent whose token created it — no cross-Agent visibility. ## Checking Agent status ```bash curl $GTI_HUB_URL/v1/agents/me -H "Authorization: Bearer $AGENT_TOKEN" ``` Returns `agent_id`, `company_id`, `enabled`, `last_seen_at`, `last_status`, `schema_fingerprint`, `agent_version`, and a derived `status` of `ONLINE` or `OFFLINE` (offline if no heartbeat arrived within `AGENT_ONLINE_THRESHOLD_SECONDS`, default 120s — configurable on the Hub). No dashboard yet; this is the query an operator or a future dashboard would poll. ## Write safety: enabling FIREBIRD_WRITES for a real client **Do not set `FIREBIRD_WRITES=1` against a real client database casually.** The authorization model has two independent layers, both required: 1. **Path**: `AUTHORIZED_WRITE_DATABASE_PATH` in that installation's `agent/.env` must be set to the *exact* absolute path of `FIREBIRD_DATABASE_PATH` for that same installation. If it's unset, the guard falls back to the developer-only disposable test database path (`TESTESOFICIAIS.FDB`), so a real client path is refused automatically — authorization is never inferred from the filename or the `.FDB` extension. 2. **Schema fingerprint**: `EXPECTED_SCHEMA_FINGERPRINT` must equal the live schema fingerprint that installation reports in its heartbeat (`schema_fingerprint`, visible via `GET /v1/agents/me`). If the live schema ever drifts from what was verified, writes are refused until an operator re-verifies and updates this value. Both checks happen before any write connection is opened (`scripts/firebird_write_executor.py::assert_authorized_write_path` / `assert_schema_fingerprint_authorized`), and before any product is even read from Firebird for that job. Even with both layers satisfied, auto-apply only ever touches `C000025.IPI`, and only when the TIPI match is `rate_kind=NUMERIC` and unambiguous. **No real client database has been written to during this or any prior mission** — all writes so far were against `TESTESOFICIAIS.FDB`. ## Logs and diagnostics - **Agent**: rotating local log file (`AGENT_LOG_PATH` in `agent/.env`, default `agent/logs/agent.log`, 5MB × 5 backups). Contains startup, heartbeat outcomes, job received/finished, products processed, write counts, failures, and Hub reconnects — never secrets (`FB_PASSWORD` and `AGENT_TOKEN` are never logged). Detailed per-product audit is in the Agent's SQLite file (`AGENT_STATE_DB`, table `ipi_write_log`). - **Hub**: structured `logging` output (visible via `docker compose logs fiscal-hub`) for startup, health-check failures, heartbeats, and the full Job lifecycle (created/claimed/completed/failed, with sanitized errors). Coarse per-Job audit persists in Postgres (`jobs.result_summary`). ## Troubleshooting | Symptom | Check | |---|---| | Agent shows `OFFLINE` | Is the service running (`sc.exe query GTIFiscalAgent`)? Check `agent/logs/agent.log` for heartbeat failures — usually `GTI_HUB_URL` unreachable or wrong `AGENT_TOKEN`. | | Job stuck `PENDING` | Confirm the Agent that owns it is running and polling (its `agent_id` must match the Job's `agent_id` — an Agent never claims another Agent's Job). | | Job `FAILED` | `GET /v1/jobs/{id}` for the sanitized `error`. Common causes: Firebird unreachable, `FIREBIRD_WRITES=1` misconfigured (see "Write safety"), Hub unreachable at report time (the Agent keeps the local `ipi_write_log` regardless). | | `writes_attempted` always 0 with `FIREBIRD_WRITES=1` | Check both `AUTHORIZED_WRITE_DATABASE_PATH` and `EXPECTED_SCHEMA_FINGERPRINT` are set and correct for that installation. | | Hub `/health` returns 503 | Postgres is down or unreachable; check `docker compose ps` and `docker compose logs postgres`. | ## Rollback / uninstalling an Agent 1. Stop and remove the Windows service (see above). 2. Delete the `agent/` installation directory on that machine (its SQLite audit and logs are local-only; nothing to clean up centrally). 3. On the Hub, disable the Agent rather than deleting history: set `enabled = false` for that `agents` row (no HTTP endpoint for this yet — run it via `psql`/a short script against `DATABASE_URL`, the same access `register_agent.py` uses). Its past Jobs and `classification_cache` entries remain for audit. ## Rolling back the Hub `docker compose down` does not remove the `postgres_data` volume — data survives. To roll back to a previous Hub image/version, redeploy that version's code and run `alembic upgrade head` again (migrations are additive, so this is safe going forward; there is no tested downgrade path for `0006`, so don't `alembic downgrade` in a pilot without restoring from a backup first).