Add 'fiscal/' from commit '102ba84500dbd9655a0471288c3e278652593ca9'
git-subtree-dir: fiscal git-subtree-mainline:d95165fc8fgit-subtree-split:102ba84500
This commit is contained in:
@@ -0,0 +1,505 @@
|
||||
# GTI Fiscal — Pilot Deployment Guide
|
||||
|
||||
Living document for operating a real pilot: one Hub, one or more client
|
||||
Agent installations. See `docs/OPERATIONS.md` for the day-to-day command
|
||||
reference (creating jobs, checking status); this document is about getting a
|
||||
new environment stood up and kept alive.
|
||||
|
||||
## RC1 (1.1.0) — supported path
|
||||
|
||||
```
|
||||
Hub central (Docker: Hub + PostgreSQL, deploy/hub)
|
||||
^ HTTPS (TLS terminated by a reverse proxy)
|
||||
|
|
||||
Agent Windows (service "GTIFiscalAgent", deploy/agent-windows)
|
||||
| local read-only connection
|
||||
SGV Firebird (the client's C000025 / C000004)
|
||||
```
|
||||
|
||||
The Hub **never** accesses Firebird, never stores Firebird credentials and never holds the client's database path. Only the Agent, on the client's Windows host, opens
|
||||
Firebird. RC1 pilots run `FISCAL_MODE=DRY_RUN` + `FIREBIRD_WRITES=0`: the Agent is technically unable to write (no write connection is ever opened, the executor is never called).
|
||||
|
||||
### Hub (GTI infrastructure)
|
||||
1. `cd deploy/hub && cp .env.example .env` — set `POSTGRES_PASSWORD` (`openssl rand -hex 24`), `BASE_URL` (public HTTPS URL). Leave `HUB_BIND=127.0.0.1` and publish it through the reverse proxy.
|
||||
2. `docker compose -f docker-compose.prod.yml --env-file .env up -d --build` — PostgreSQL (internal network only, persistent volume, healthcheck), Hub (migrations at start, healthcheck, `restart: unless-stopped`, json-file logs 10 MB x 5).
|
||||
3. Seed the validated datasets once: on the development machine `sh export_datasets_seed.sh > gti_datasets_seed.dump`; on the Hub host `sh seed_datasets.sh gti_datasets_seed.dump` (refuses if `datasets` is not empty; reference tables only, never tenants/agents/jobs).
|
||||
4. Register the Agent: `docker compose -f docker-compose.prod.yml exec fiscal-hub python central/scripts/register_agent.py --company "<Client>" --name "<installation>"` — the token is printed once; store it only in the Agent's local `.env`.
|
||||
5. Health: `GET /health` (Postgres included), `GET /version` (release/engine/decision versions). `APP_ENV=production` refuses to start when `AGENT_DEV_TOKEN` is set.
|
||||
6. Backup/restore: `sh backup.sh` (pg_dump -Fc into `BACKUP_DIR`, keeps 14) and `sh restore.sh <dump>` (asks for the word RESTORE, stops the Hub, restores, restarts; migrations rerun at start).
|
||||
|
||||
### Agent (client host, PowerShell as Administrator, inside the extracted `GTIFiscalAgent-<version>` folder)
|
||||
Build the package on the release machine: `python scripts/build_agent_package.py` -> `dist/GTIFiscalAgent-<version>.zip` (the version is `release/VERSION`) (+ `.sha256`; secret-scanned, no `.env`).
|
||||
|
||||
| step | command |
|
||||
|---|---|
|
||||
| install | `.\gti-agent.ps1 install` (venv + dependencies + service registration; does not start) |
|
||||
| configure | `.\gti-agent.ps1 configure` (prompts; secrets never echoed) or `configure -FromTemplate .\pilot-construcao.env.example` |
|
||||
| preflight | `.\gti-agent.ps1 preflight` -> `PREFLIGHT=PASS|FAIL` + blockers |
|
||||
| start / stop / restart | `.\gti-agent.ps1 start` (refuses when preflight fails) |
|
||||
| status | `.\gti-agent.ps1 status` |
|
||||
| uninstall | `.\gti-agent.ps1 uninstall [-Purge]` |
|
||||
|
||||
### Agent configuration (per installation, `.env` in the install folder)
|
||||
`GTI_HUB_URL`, `AGENT_ID`, `AGENT_TOKEN`; `COMPANY_CODE` / `COMPANY_CNPJ` (company selection), `COMPANY_CRT_SOURCE` = `AUTO` | `EMISSOR` | `C000004`; `COMPANY_UF` only as an explicit
|
||||
override (it never replaces the resolver: it must agree with what the database proves); `FIREBIRD_DATABASE_PATH`, `FB_USER` (read-only account), `FB_PASSWORD`; `FISCAL_MODE=DRY_RUN`, `FIREBIRD_WRITES=0`;
|
||||
`POLL_INTERVAL_SECONDS`, `AGENT_VERSION`. `AUTO` resolves the CRT only when EMISSOR and C000004 agree; a conflict makes the preflight FAIL until `EMISSOR` or `C000004` is chosen explicitly.
|
||||
The first row of C000004 is never used to pick the company.
|
||||
|
||||
### Preflight (`agent/scripts/preflight.py`, also enforced at service start)
|
||||
HUB_REACHABLE, AUTH, HUB_VERSION_COMPATIBLE (major.minor), FIREBIRD_REACHABLE, FIREBIRD_VERSION_ODS, C000004, C000025, CAPABILITIES (required: NCM, IPI, CEST), COMPANY_CONTEXT (UF, CRT),
|
||||
SCHEMA_TYPE_GUARDS, SQLITE, LOG_DIRECTORY, FISCAL_MODE, FIREBIRD_WRITES (`DRY_RUN` + `FIREBIRD_WRITES=1` is a misconfiguration -> FAIL).
|
||||
|
||||
### Pilot profiles
|
||||
`deploy/agent-windows/pilot-construcao.env.example` (first pilot: company 000001, UF BA, CRT 1, `COMPANY_CRT_SOURCE=AUTO`, DRY_RUN). Planned, **not installed**: `profiles/mmmodas|rincao|uniao.env.example`
|
||||
(`COMPANY_CRT_SOURCE=EMISSOR` required: EMISSOR and C000004 disagree on the CRT). Never reuse the path of a local database copy as the client's real path.
|
||||
|
||||
### Zero-write proof
|
||||
`python scripts/prove_zero_write.py snapshot <FDB> before.json` ... run the pilot ... `snapshot <FDB> after.json` ... `compare before.json after.json` (database DML counters, SHA-256 of every row of C000025/C000004, row counts).
|
||||
|
||||
---
|
||||
|
||||
The sections below are the previous manual instructions (still accurate for development); the RC1 scripts above are the supported way to deploy.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌────────────────────────────┐
|
||||
│ Hub server (GTI/Proxmox) │
|
||||
│ Docker Compose: │
|
||||
│ - postgres (data volume) │
|
||||
│ - fiscal-hub (FastAPI) │
|
||||
└─────────────┬────────────────┘
|
||||
│ HTTPS (reverse proxy terminates TLS)
|
||||
┌────────────────────────┼────────────────────────┐
|
||||
│ │ │
|
||||
┌─────────▼─────────┐ ┌─────────▼─────────┐ ┌─────────▼─────────┐
|
||||
│ Client A — Agent │ │ Client B — Agent │ │ Client C — Agent │
|
||||
│ Windows Service │ │ Windows Service │ │ Windows Service │
|
||||
│ + local Firebird │ │ + local Firebird │ │ + local Firebird │
|
||||
│ + local SQLite audit│ │ + local SQLite audit│ │ + local SQLite audit│
|
||||
└────────────────────┘ └────────────────────┘ └────────────────────┘
|
||||
```
|
||||
|
||||
The Hub never has Firebird credentials and never opens a Firebird
|
||||
connection. Each Agent has its own token and only ever reads/writes the one
|
||||
Firebird database it was configured for.
|
||||
|
||||
## Hub server requirements
|
||||
|
||||
- Docker + Docker Compose v2.
|
||||
- Enough disk for the Postgres volume (starts small; grows with
|
||||
`classification_cache`/`jobs` history — no large binary data).
|
||||
- One inbound port for the Hub API (default `8000`, or whatever your reverse
|
||||
proxy forwards to it). Postgres itself does **not** need a host port —
|
||||
`docker-compose.yml` intentionally does not publish `5432`; only containers
|
||||
on the compose network reach it.
|
||||
|
||||
## Deploying the Hub
|
||||
|
||||
```bash
|
||||
git clone <repo> && cd Projeto
|
||||
cp .env.example .env
|
||||
# Edit .env: set a real POSTGRES_PASSWORD, leave AGENT_DEV_TOKEN empty,
|
||||
# set APP_ENV=production, BASE_URL to the public HTTPS URL.
|
||||
docker compose up -d postgres fiscal-hub
|
||||
curl http://localhost:8000/health # {"status": "ok"}
|
||||
```
|
||||
|
||||
`fiscal-hub`'s container command runs `alembic upgrade head` before starting
|
||||
`uvicorn`, so schema migrations apply automatically on every deploy/restart.
|
||||
Both services have `restart: unless-stopped` and Docker healthchecks
|
||||
(`pg_isready` for Postgres, an HTTP GET to `/health` for the Hub).
|
||||
|
||||
**Never** put `AGENT_DEV_TOKEN` in a production `.env` — it auto-bootstraps
|
||||
an agent from a fixed shared token on every request, fine for local dev,
|
||||
wrong for a pilot with real client secrets. Leave it empty; register real
|
||||
Agents with `central/scripts/register_agent.py` (see below).
|
||||
|
||||
### Persistence
|
||||
|
||||
- Postgres data lives in the named volume `postgres_data`. Back it up with
|
||||
`docker exec <postgres-container> pg_dump -U gti gti_fiscal > backup.sql`
|
||||
(or `pg_dumpall` for the whole cluster) on whatever schedule your
|
||||
operations policy requires.
|
||||
- Always run `alembic upgrade head` (the container does this automatically)
|
||||
rather than hand-editing the schema. An **empty** database is initialized
|
||||
by that same command — never with `alembic stamp` or `create_all`. `0001` is
|
||||
an explicit, frozen baseline (it used to call `Base.metadata.create_all()`,
|
||||
which made `0002` collide on `datasets.raw_sha256` on any fresh database);
|
||||
`central/tests/test_migration_baseline.py` keeps that from coming back.
|
||||
- `0019` repairs `agents.created_at/updated_at` and `jobs.created_at`, whose
|
||||
defaults `0006` had frozen at the instant it ran (rows created before `0019`
|
||||
keep that frozen value — their real creation time is not recoverable).
|
||||
- `0020` enforces NOT NULL on 20 timestamp columns that always had
|
||||
`DEFAULT now()`. It aborts, changing nothing, if any NULL exists — clean
|
||||
those rows explicitly first. `0021` adds the SSO federated identity (see "Human
|
||||
access"); `0022` adds Agent releases + secure enrollment (see "Agent
|
||||
distribution"). Current head: `0022_agent_releases_enrollment`
|
||||
(also recorded in `release/RELEASE_MANIFEST.json`).
|
||||
|
||||
### HTTPS / reverse proxy
|
||||
|
||||
The Hub itself speaks plain HTTP. For a pilot reachable outside the Hub's
|
||||
own host, put it behind a reverse proxy (nginx, Caddy, Traefik — whatever
|
||||
the GTI/Proxmox environment already standardizes on) that terminates TLS and
|
||||
forwards to `fiscal-hub:8000`. Nothing in the app assumes a particular
|
||||
proxy; `BASE_URL` in `.env` is informational only (not currently used to
|
||||
generate absolute links, but keep it accurate for anyone reading the config).
|
||||
|
||||
**GTI production (2026-09-23):** TLS terminates in the GTI Nginx Proxy Manager
|
||||
(10.0.10.100), which forwards `http://<hub-lan-ip>:8000`. On the Hub host:
|
||||
|
||||
- `.env`: `HUB_BIND=<hub LAN IP>` (not `0.0.0.0`), `FORWARDED_ALLOW_IPS=<NPM IP>`
|
||||
(real client IP and `https` scheme from `X-Forwarded-*`), `BASE_URL=https://…`,
|
||||
and `WEB_COOKIE_SECURE=1` once HTTPS works. Redirects are relative, so the
|
||||
internal address never leaks.
|
||||
- Docker-published ports bypass UFW: install `restrict_hub_port.sh` +
|
||||
`gti-fiscal-hub-port.service` (instructions in the unit file) so only the
|
||||
NPM reaches the port, re-applied on every Docker start.
|
||||
|
||||
## Human access: SSO from GTI Gerenciador
|
||||
|
||||
Collaborators never type a second password. GTI Gerenciador (the panel) is the
|
||||
identity authority; the Hub keeps its own sessions.
|
||||
|
||||
1. "Fiscal" in the panel sidebar -> panel `/sso/fiscal` -> Hub `GET /login`,
|
||||
which sets a one-time state cookie (`gti_sso_state`, HttpOnly, Secure,
|
||||
SameSite=None, 5 min) and redirects to `SSO_START_URL?state=...&next=...`.
|
||||
2. The panel requires its own login (normal login page if needed), checks the
|
||||
employee is active and auto-POSTs a JWT to `/auth/sso/callback`:
|
||||
Ed25519 (`EdDSA` only), `iss=gti-gerenciador`, `aud=gti-fiscal`,
|
||||
`sub=funcionario:<ID>`, 60 s, unique `jti`, `nonce`=state. Never in a URL.
|
||||
3. The Hub verifies signature/iss/aud/exp/nbf/lifetime, the nonce against the
|
||||
state cookie (login-CSRF protection), records the `jti`
|
||||
(`sso_used_assertions`, replay = rejected), provisions or refreshes the
|
||||
collaborator in `admin_users` (`auth_source=GTI_GERENCIADOR`, no password,
|
||||
enforced by `ck_admin_users_auth_source`) and opens a session of
|
||||
`FEDERATED_SESSION_SECONDS` (default 1 h; afterwards the same flow signs the
|
||||
user in again silently, which also picks up a deactivation in the panel).
|
||||
|
||||
Keys: a dedicated Ed25519 pair, used for nothing else (never reuse
|
||||
SESSION_SECRET, admin, Agent or DB secrets). The **private** key
|
||||
(raw 32-byte seed, base64url) goes only to the panel's `.env`
|
||||
(`SSO_FISCAL_CHAVE_PRIVADA`); the **public** key only to this Hub's `.env`
|
||||
(`SSO_GERENCIADOR_CHAVE_PUBLICA`). Rotating = new pair in both, restart both.
|
||||
|
||||
Logout ("Sair do Fiscal") ends only the Hub session; coming back through the
|
||||
panel signs in again without a password. There is no distributed single logout.
|
||||
|
||||
Audit (`audit_events`): `SSO_LOGIN_SUCCESS`, `SSO_LOGIN_REJECTED` (with a stable
|
||||
reason: `EXPIRED`, `INVALID_SIGNATURE`, `WRONG_AUDIENCE`, `WRONG_ISSUER`,
|
||||
`REPLAY`, `STATE_MISMATCH`, `DISABLED_IN_FISCAL`, ...), `LOGOUT`,
|
||||
`LOGIN_LOCAL_SUCCESS/FAILURE` — with the real client IP, never the token.
|
||||
|
||||
**Break-glass:** the local admin (`create_admin.py`) still exists for
|
||||
emergencies at `/login/local` (not linked from the UI; every use is audited).
|
||||
Disable a collaborator only in the Hub: `admin_users.is_active = false`.
|
||||
|
||||
## Registering an Agent (per client installation)
|
||||
|
||||
Run once per Agent installation, from anywhere with access to Postgres —
|
||||
easiest is inside the running Hub container:
|
||||
|
||||
```bash
|
||||
docker exec <fiscal-hub-container> python central/scripts/register_agent.py \
|
||||
--company "Client Company Name" --name "loja-01"
|
||||
```
|
||||
|
||||
This prints `AGENT_ID`, a freshly generated `AGENT_TOKEN` (shown once — copy
|
||||
it immediately), and `COMPANY_ID`. Only the token's SHA-256 hash is ever
|
||||
stored in Postgres (same `token_hash` mechanism `/v1/classify` already used
|
||||
— no new auth system). Each Agent gets its own token; one Agent's token
|
||||
never authenticates another Agent's jobs (`agent_id` is checked on every
|
||||
Job operation).
|
||||
|
||||
## Agent distribution: install from the panel (secure enrollment)
|
||||
|
||||
An alternative to `register_agent.py` for a real pilot: the panel drives the
|
||||
whole installation, and the technician on the client's Windows host never
|
||||
gets a permanent secret by hand.
|
||||
|
||||
**One-time: publish a release.** The Agent's universal package (one zip per
|
||||
version — never one per client) is built and published by script, never
|
||||
uploaded through a browser:
|
||||
|
||||
```bash
|
||||
python scripts/build_agent_package.py # -> dist/GTIFiscalAgent-<version>.zip
|
||||
docker exec <fiscal-hub-container> python central/scripts/publish_agent_release.py \
|
||||
--zip dist/GTIFiscalAgent-<version>.zip --notes "..."
|
||||
```
|
||||
|
||||
Publishing the same version again with different bytes is refused (bump the
|
||||
version instead); publishing identical bytes again is a no-op. The zip lives
|
||||
on the Hub's local `agent_releases` volume (`AGENT_RELEASE_STORAGE_PATH`,
|
||||
default `/app/data/releases`), never in PostgreSQL — only its metadata
|
||||
(`agent_releases`: version, sha256, size, status DRAFT/PUBLISHED/REVOKED) is
|
||||
a row. `Agents / Servidores -> Releases` lists and can revoke a release.
|
||||
|
||||
**Per installation:** `Agents / Servidores -> + Nova instalação` — pick the
|
||||
client, a display name, an optional expected hostname and the release,
|
||||
"Criar instalação". That installation's page then offers:
|
||||
|
||||
- **Baixar Agent agora** — an authenticated, immediate download (the operator
|
||||
is already logged in).
|
||||
- **Gerar link temporário** — a one-time HTTPS URL
|
||||
(`/dl/agent/<download-token>`), shown exactly once, TTL
|
||||
`AGENT_DOWNLOAD_LINK_TTL_SECONDS` (default 30 min), single use
|
||||
(`AGENT_DOWNLOAD_MAX_USES_DEFAULT`, default 1). Hand it to a technician or
|
||||
open it directly on the client's server — no login required to use it.
|
||||
|
||||
Either path streams a bundle = the universal package + a freshly minted
|
||||
`enrollment.json` (installation id, a one-time enrollment token, the Hub URL)
|
||||
written at download time. Only that enrollment token's SHA-256 hash is ever
|
||||
stored (`agent_enrollments.enrollment_token_hash`); its plaintext exists only
|
||||
inside that one HTTP response — never on disk on the Hub, never logged.
|
||||
|
||||
**On the client Windows host**, elevated PowerShell, inside the extracted
|
||||
package:
|
||||
|
||||
```powershell
|
||||
.\gti-agent.ps1 install-auto
|
||||
```
|
||||
|
||||
This checks every file in the bundle against `MANIFEST.json`'s SHA-256,
|
||||
refuses a non-HTTPS Hub URL, calls `POST /v1/agents/enroll` with the
|
||||
enrollment token (deleting `enrollment.json` immediately after, success or
|
||||
failure), writes `.env` with the newly issued **permanent** `AGENT_TOKEN`
|
||||
(ACL-restricted to Administrators/SYSTEM, same as `configure`), and
|
||||
registers the Windows Service — **without starting it**: Firebird is still
|
||||
unconfigured on purpose (this script never guesses a client's database path
|
||||
or credentials, same rule as the manual path). Edit `FIREBIRD_DATABASE_PATH`
|
||||
/ `FB_USER` / `FB_PASSWORD` / `COMPANY_CODE` in `.env`, then
|
||||
`.\gti-agent.ps1 preflight ; .\gti-agent.ps1 start`. Running `install-auto`
|
||||
again after a successful enrollment is a no-op (idempotent).
|
||||
|
||||
The Hub rejects: an expired or already-used download link, a revoked
|
||||
installation or release, a wrong/tampered/replayed enrollment token, and a
|
||||
second enrollment attempt for an already-enrolled installation. A hostname
|
||||
that doesn't match what the operator expected is recorded
|
||||
(`hostname_mismatch`) and audited, but does not block enrollment. Both
|
||||
public routes (`GET /dl/agent/{token}`, `POST /v1/agents/enroll`) are
|
||||
rate-limited per source IP. Audit actions: `INSTALLATION_CREATED`,
|
||||
`DOWNLOAD_LINK_CREATED`, `DOWNLOAD_LINK_REJECTED`, `PACKAGE_DOWNLOADED`,
|
||||
`ENROLLMENT_SUCCEEDED`, `ENROLLMENT_REJECTED`, `AGENT_CREDENTIAL_ISSUED`,
|
||||
`INSTALLATION_REVOKED`, `AGENT_RELEASE_REVOKED` — the real IP, never a token.
|
||||
|
||||
Revoking a pending or enrolled installation (`Revogar` on its detail page)
|
||||
invalidates any outstanding link/enrollment token immediately and, if an
|
||||
Agent was already provisioned, disables it (`agents.enabled = false`) — its
|
||||
existing sessions/jobs stop authenticating on the next request.
|
||||
|
||||
## Configuring an Agent installation
|
||||
|
||||
On the client machine, copy `agent/.env.example` to `agent/.env` and fill in
|
||||
the values from registration plus that machine's Firebird details:
|
||||
|
||||
```
|
||||
GTI_HUB_URL=https://hub.example.com
|
||||
AGENT_ID=<from registration>
|
||||
AGENT_TOKEN=<from registration>
|
||||
COMPANY_ID=<from registration>
|
||||
COMPANY_UF=BA
|
||||
FIREBIRD_DATABASE_PATH=C:\ERP\dados\PRODUCAO.FDB
|
||||
FB_USER=...
|
||||
FB_PASSWORD=...
|
||||
FIREBIRD_WRITES=0
|
||||
POLL_INTERVAL_SECONDS=10
|
||||
AGENT_VERSION=0.1.0
|
||||
```
|
||||
|
||||
**`FIREBIRD_WRITES` must default to `0`.** See "Write safety" below before
|
||||
ever setting it to `1`.
|
||||
|
||||
## Installing the Agent as a Windows Service
|
||||
|
||||
Requires `pywin32` (`pip install pywin32`, or `pip install -e ".[windows-service]"`
|
||||
from `agent/`).
|
||||
|
||||
```powershell
|
||||
# From an elevated (Administrator) terminal, in agent/service/:
|
||||
python gti_agent_windows_service.py --startup=auto install
|
||||
python gti_agent_windows_service.py start
|
||||
|
||||
# One-time, so Windows restarts it automatically after a crash:
|
||||
sc.exe failure "GTIFiscalAgent" reset= 86400 actions= restart/60000/restart/60000/restart/60000
|
||||
```
|
||||
|
||||
`gti_agent_windows_service.py` is a thin wrapper: it does not reimplement
|
||||
polling, heartbeat, or the pipeline — it calls `run_agent_service.run()`
|
||||
directly, stopping it via the Service Control Manager's own stop signal
|
||||
instead of `SIGINT`/`SIGTERM` (which don't apply to services running with no
|
||||
console). See the module's docstring for the exact commands.
|
||||
|
||||
### Install / configure / start / status / stop / restart / uninstall
|
||||
|
||||
```powershell
|
||||
# Install (registers with SCM, Startup=Automatic)
|
||||
python gti_agent_windows_service.py --startup=auto install
|
||||
|
||||
# Configure: edit agent/.env, no service restart needed until you `restart`
|
||||
notepad agent\.env
|
||||
|
||||
# Start
|
||||
python gti_agent_windows_service.py start
|
||||
|
||||
# Status
|
||||
sc.exe query GTIFiscalAgent
|
||||
|
||||
# Stop
|
||||
python gti_agent_windows_service.py stop
|
||||
|
||||
# Restart (picks up any agent/.env changes)
|
||||
python gti_agent_windows_service.py restart
|
||||
|
||||
# Uninstall
|
||||
python gti_agent_windows_service.py stop
|
||||
python gti_agent_windows_service.py remove
|
||||
```
|
||||
|
||||
### Updating the Agent's version
|
||||
|
||||
1. Stop the service: `python gti_agent_windows_service.py stop`
|
||||
2. Replace the `agent/` code with the new version (git pull, or copy files).
|
||||
3. Bump `AGENT_VERSION` in `agent/.env` if you track it per-release.
|
||||
4. Start the service again: `python gti_agent_windows_service.py start`.
|
||||
5. Confirm the new version shows up via `GET /v1/agents/me` on the Hub.
|
||||
|
||||
There is no auto-update mechanism yet (deliberately out of scope for this
|
||||
pilot) — updates are a manual stop/replace/start.
|
||||
|
||||
### Running without installing as a service (dev/testing)
|
||||
|
||||
```bash
|
||||
cd agent
|
||||
python scripts/run_agent_service.py # runs forever, Ctrl+C to stop
|
||||
python scripts/run_agent_service.py --max-iterations 5 # bounded, for scripting/tests
|
||||
```
|
||||
|
||||
## Creating and checking work
|
||||
|
||||
See `docs/OPERATIONS.md` for the full `curl` examples
|
||||
(`POST /v1/jobs` with `kind: REPROCESS_ALL` or `REPROCESS_PRODUCTS`,
|
||||
`GET /v1/jobs/{id}` for status). Every Job is scoped to the Agent whose
|
||||
token created it — no cross-Agent visibility.
|
||||
|
||||
## Checking Agent status
|
||||
|
||||
```bash
|
||||
curl $GTI_HUB_URL/v1/agents/me -H "Authorization: Bearer $AGENT_TOKEN"
|
||||
```
|
||||
|
||||
Returns `agent_id`, `company_id`, `enabled`, `last_seen_at`, `last_status`,
|
||||
`schema_fingerprint`, `agent_version`, and a derived `status` of `ONLINE` or
|
||||
`OFFLINE` (offline if no heartbeat arrived within
|
||||
`AGENT_ONLINE_THRESHOLD_SECONDS`, default 120s — configurable on the Hub).
|
||||
No dashboard yet; this is the query an operator or a future dashboard would
|
||||
poll.
|
||||
|
||||
## Write safety: enabling FIREBIRD_WRITES for a real client
|
||||
|
||||
**Do not set `FIREBIRD_WRITES=1` against a real client database casually.**
|
||||
The authorization model has two independent layers, both required:
|
||||
|
||||
1. **Path**: `AUTHORIZED_WRITE_DATABASE_PATH` in that installation's
|
||||
`agent/.env` must be set to the *exact* absolute path of
|
||||
`FIREBIRD_DATABASE_PATH` for that same installation. If it's unset, the
|
||||
guard falls back to the developer-only disposable test database path
|
||||
(`TESTESOFICIAIS.FDB`), so a real client path is refused automatically —
|
||||
authorization is never inferred from the filename or the `.FDB`
|
||||
extension.
|
||||
2. **Schema fingerprint**: `EXPECTED_SCHEMA_FINGERPRINT` must equal the live
|
||||
schema fingerprint that installation reports in its heartbeat
|
||||
(`schema_fingerprint`, visible via `GET /v1/agents/me`). If the live
|
||||
schema ever drifts from what was verified, writes are refused until an
|
||||
operator re-verifies and updates this value.
|
||||
|
||||
Both checks happen before any write connection is opened
|
||||
(`scripts/firebird_write_executor.py::assert_authorized_write_path` /
|
||||
`assert_schema_fingerprint_authorized`), and before any product is even
|
||||
read from Firebird for that job.
|
||||
|
||||
Even with both layers satisfied, auto-apply only ever touches
|
||||
`C000025.IPI`, and only when the TIPI match is `rate_kind=NUMERIC` and
|
||||
unambiguous. **No real client database has been written to during this or
|
||||
any prior mission** — all writes so far were against `TESTESOFICIAIS.FDB`.
|
||||
|
||||
## Logs and diagnostics
|
||||
|
||||
- **Agent**: rotating local log file (`AGENT_LOG_PATH` in `agent/.env`,
|
||||
default `agent/logs/agent.log`, 5MB × 5 backups). Contains startup,
|
||||
heartbeat outcomes, job received/finished, products processed, write
|
||||
counts, failures, and Hub reconnects — never secrets (`FB_PASSWORD` and
|
||||
`AGENT_TOKEN` are never logged). Detailed per-product audit is in the
|
||||
Agent's SQLite file (`AGENT_STATE_DB`, table `ipi_write_log`).
|
||||
- **Hub**: structured `logging` output (visible via `docker compose logs
|
||||
fiscal-hub`) for startup, health-check failures, heartbeats, and the full
|
||||
Job lifecycle (created/claimed/completed/failed, with sanitized errors).
|
||||
Coarse per-Job audit persists in Postgres (`jobs.result_summary`).
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Check |
|
||||
|---|---|
|
||||
| Agent shows `OFFLINE` | Is the service running (`sc.exe query GTIFiscalAgent`)? Check `agent/logs/agent.log` for heartbeat failures — usually `GTI_HUB_URL` unreachable or wrong `AGENT_TOKEN`. |
|
||||
| Job stuck `PENDING` | Confirm the Agent that owns it is running and polling (its `agent_id` must match the Job's `agent_id` — an Agent never claims another Agent's Job). |
|
||||
| Job `FAILED` | `GET /v1/jobs/{id}` for the sanitized `error`. Common causes: Firebird unreachable, `FIREBIRD_WRITES=1` misconfigured (see "Write safety"), Hub unreachable at report time (the Agent keeps the local `ipi_write_log` regardless). |
|
||||
| `writes_attempted` always 0 with `FIREBIRD_WRITES=1` | Check both `AUTHORIZED_WRITE_DATABASE_PATH` and `EXPECTED_SCHEMA_FINGERPRINT` are set and correct for that installation. |
|
||||
| Hub `/health` returns 503 | Postgres is down or unreachable; check `docker compose ps` and `docker compose logs postgres`. |
|
||||
|
||||
## Rollback / uninstalling an Agent
|
||||
|
||||
1. Stop and remove the Windows service (see above).
|
||||
2. Delete the `agent/` installation directory on that machine (its SQLite
|
||||
audit and logs are local-only; nothing to clean up centrally).
|
||||
3. On the Hub, disable the Agent rather than deleting history: set
|
||||
`enabled = false` for that `agents` row (no HTTP endpoint for this yet —
|
||||
run it via `psql`/a short script against `DATABASE_URL`, the same access
|
||||
`register_agent.py` uses). Its past Jobs and `classification_cache`
|
||||
entries remain for audit.
|
||||
|
||||
## Rolling back the Hub
|
||||
|
||||
`docker compose down` does not remove the `postgres_data` volume — data
|
||||
survives. To roll back to a previous Hub image/version, redeploy that
|
||||
version's code and run `alembic upgrade head` again (migrations are
|
||||
additive, so this is safe going forward; there is no tested downgrade path
|
||||
for `0006`, so don't `alembic downgrade` in a pilot without restoring from a
|
||||
backup first).
|
||||
|
||||
## Zero-touch Agent (Setup 1.1.3+): install once, control from the panel
|
||||
|
||||
The operator runs `GTIFiscalAgent-Setup-<version>.exe` **once** and types the install
|
||||
code. Nothing else happens on the client server — no PowerShell, no `agent.env` editing,
|
||||
no `services.msc`, no per-client cron:
|
||||
|
||||
1. **Firebird discovery (read-only).** Setup finds the Firebird service and candidate
|
||||
`.FDB` files and probes each one (read-only connection) for `C000025` + `C000004`.
|
||||
Exactly one SGV database -> selected automatically. Several, none, or credentials
|
||||
other than the SGV default -> the wizard asks *during the installation* (list / file
|
||||
picker + user/password, verified before use). It never guesses among several.
|
||||
2. **Enrollment** with the install code, then three protected files (DPAPI LocalMachine +
|
||||
SYSTEM/Administrators ACL): `credentials.bin` (Hub token), `firebird.bin` (Firebird
|
||||
user/password, **never sent to the Hub**), and a non-secret `agent.env`.
|
||||
3. **Service** `GTIFiscalAgent`: Automatic Delayed Start, restart-on-failure, **started by
|
||||
Setup**; it waits for the first heartbeat and tells the operator "nenhuma outra ação".
|
||||
4. **First sync is automatic.** On the first heartbeat of an installation without
|
||||
coverage the Hub queues exactly one `INITIAL_SYNC` (idempotent). Afterwards the Hub's
|
||||
scheduler queues one `SYNC_INCREMENTAL` per interval (default 30 min,
|
||||
`installations.sync_interval_seconds`), only when nothing is queued/running.
|
||||
`SYNC_INCREMENTAL` re-classifies only new/changed products (per-product source
|
||||
fingerprint cache in the Agent's SQLite) and rebuilds coverage from the cache.
|
||||
`REPROCESS_ALL` runs only when an operator asks for it.
|
||||
5. **Central control** (Agent detail page, no shell, no arbitrary commands):
|
||||
*Sincronizar agora*, *Reprocessar tudo*, *Pausar / Retomar processamento*, plus status
|
||||
(Ativo/Pausado, última/agendada sincronização, produtos alterados, último job).
|
||||
Pause = the heartbeat keeps flowing but the Hub hands out no job and schedules none.
|
||||
The old local `AGENT_JOBS_ENABLED` is ignored — there is no local toggle.
|
||||
|
||||
Guarantees: one fiscal job per Agent at a time (FIFO queue, a job stuck RUNNING > 1 h is
|
||||
failed so it cannot block the queue), a failed sync backs off for the interval instead of
|
||||
looping, a Hub or Firebird outage fails the job cleanly and the service keeps running.
|
||||
Every job kind is **read-only on the ERP** (`FIREBIRD_WRITES=0`, `FISCAL_MODE=DRY_RUN`).
|
||||
|
||||
Upgrading an installed host: run the newer Setup again (no code needed). It replaces the
|
||||
binaries, keeps `agent_id`/`installation_id`/credentials, removes the legacy local toggle and
|
||||
restarts the service.
|
||||
Reference in New Issue
Block a user