gustavosantos/GTI_GERENCIADOR_FISCAL
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: establish project baseline

Browse Source
This commit is contained in:
OpenCode
2026-09-21 15:51:47 -03:00
commit 9dcc263217
459 changed files with 447890 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PILOT_DEPLOYMENT.md
+330
View File
@@ -0,0 +1,330 @@
# GTI Fiscal — Pilot Deployment Guide
Living document for operating a real pilot: one Hub, one or more client
Agent installations. See `docs/OPERATIONS.md` for the day-to-day command
reference (creating jobs, checking status); this document is about getting a
new environment stood up and kept alive.
## RC1 (1.0.0-rc1) — supported path
```
Hub central (Docker: Hub + PostgreSQL, deploy/hub)
^ HTTPS (TLS terminated by a reverse proxy)
|
Agent Windows (service "GTIFiscalAgent", deploy/agent-windows)
| local read-only connection
SGV Firebird (the client's C000025 / C000004)
```
The Hub **never** accesses Firebird, never stores Firebird credentials and never holds the client's database path. Only the Agent, on the client's Windows host, opens
Firebird. RC1 pilots run `FISCAL_MODE=DRY_RUN` + `FIREBIRD_WRITES=0`: the Agent is technically unable to write (no write connection is ever opened, the executor is never called).
### Hub (GTI infrastructure)
1. `cd deploy/hub && cp .env.example .env` — set `POSTGRES_PASSWORD` (`openssl rand -hex 24`), `BASE_URL` (public HTTPS URL). Leave `HUB_BIND=127.0.0.1` and publish it through the reverse proxy.
2. `docker compose -f docker-compose.prod.yml --env-file .env up -d --build` — PostgreSQL (internal network only, persistent volume, healthcheck), Hub (migrations at start, healthcheck, `restart: unless-stopped`, json-file logs 10 MB x 5).
3. Seed the validated datasets once: on the development machine `sh export_datasets_seed.sh > gti_datasets_seed.dump`; on the Hub host `sh seed_datasets.sh gti_datasets_seed.dump` (refuses if `datasets` is not empty; reference tables only, never tenants/agents/jobs).
4. Register the Agent: `docker compose -f docker-compose.prod.yml exec fiscal-hub python central/scripts/register_agent.py --company "<Client>" --name "<installation>"` — the token is printed once; store it only in the Agent's local `.env`.
5. Health: `GET /health` (Postgres included), `GET /version` (release/engine/decision versions). `APP_ENV=production` refuses to start when `AGENT_DEV_TOKEN` is set.
6. Backup/restore: `sh backup.sh` (pg_dump -Fc into `BACKUP_DIR`, keeps 14) and `sh restore.sh <dump>` (asks for the word RESTORE, stops the Hub, restores, restarts; migrations rerun at start).
### Agent (client host, PowerShell as Administrator, inside the extracted `GTIFiscalAgent-1.0.0-rc1` folder)
Build the package on the release machine: `python scripts/build_agent_package.py` -> `dist/GTIFiscalAgent-1.0.0-rc1.zip` (+ `.sha256`; secret-scanned, no `.env`).
| step | command |
|---|---|
| install | `.\gti-agent.ps1 install` (venv + dependencies + service registration; does not start) |
| configure | `.\gti-agent.ps1 configure` (prompts; secrets never echoed) or `configure -FromTemplate .\pilot-construcao.env.example` |
| preflight | `.\gti-agent.ps1 preflight` -> `PREFLIGHT=PASS|FAIL` + blockers |
| start / stop / restart | `.\gti-agent.ps1 start` (refuses when preflight fails) |
| status | `.\gti-agent.ps1 status` |
| uninstall | `.\gti-agent.ps1 uninstall [-Purge]` |
### Agent configuration (per installation, `.env` in the install folder)
`GTI_HUB_URL`, `AGENT_ID`, `AGENT_TOKEN`; `COMPANY_CODE` / `COMPANY_CNPJ` (company selection), `COMPANY_CRT_SOURCE` = `AUTO` | `EMISSOR` | `C000004`; `COMPANY_UF` only as an explicit
override (it never replaces the resolver: it must agree with what the database proves); `FIREBIRD_DATABASE_PATH`, `FB_USER` (read-only account), `FB_PASSWORD`; `FISCAL_MODE=DRY_RUN`, `FIREBIRD_WRITES=0`;
`POLL_INTERVAL_SECONDS`, `AGENT_VERSION`. `AUTO` resolves the CRT only when EMISSOR and C000004 agree; a conflict makes the preflight FAIL until `EMISSOR` or `C000004` is chosen explicitly.
The first row of C000004 is never used to pick the company.
### Preflight (`agent/scripts/preflight.py`, also enforced at service start)
HUB_REACHABLE, AUTH, HUB_VERSION_COMPATIBLE (major.minor), FIREBIRD_REACHABLE, FIREBIRD_VERSION_ODS, C000004, C000025, CAPABILITIES (required: NCM, IPI, CEST), COMPANY_CONTEXT (UF, CRT),
SCHEMA_TYPE_GUARDS, SQLITE, LOG_DIRECTORY, FISCAL_MODE, FIREBIRD_WRITES (`DRY_RUN` + `FIREBIRD_WRITES=1` is a misconfiguration -> FAIL).
### Pilot profiles
`deploy/agent-windows/pilot-construcao.env.example` (first pilot: company 000001, UF BA, CRT 1, `COMPANY_CRT_SOURCE=AUTO`, DRY_RUN). Planned, **not installed**: `profiles/mmmodas|rincao|uniao.env.example`
(`COMPANY_CRT_SOURCE=EMISSOR` required: EMISSOR and C000004 disagree on the CRT). Never reuse the path of a local database copy as the client's real path.
### Zero-write proof
`python scripts/prove_zero_write.py snapshot <FDB> before.json` ... run the pilot ... `snapshot <FDB> after.json` ... `compare before.json after.json` (database DML counters, SHA-256 of every row of C000025/C000004, row counts).
---
The sections below are the previous manual instructions (still accurate for development); the RC1 scripts above are the supported way to deploy.
## Architecture
```
┌────────────────────────────┐
│ Hub server (GTI/Proxmox) │
│ Docker Compose: │
│ - postgres (data volume) │
│ - fiscal-hub (FastAPI) │
└─────────────┬────────────────┘
│ HTTPS (reverse proxy terminates TLS)
┌────────────────────────┼────────────────────────┐
│ │ │
┌─────────▼─────────┐ ┌─────────▼─────────┐ ┌─────────▼─────────┐
│ Client A — Agent │ │ Client B — Agent │ │ Client C — Agent │
│ Windows Service │ │ Windows Service │ │ Windows Service │
│ + local Firebird │ │ + local Firebird │ │ + local Firebird │
│ + local SQLite audit│ │ + local SQLite audit│ │ + local SQLite audit│
└────────────────────┘ └────────────────────┘ └────────────────────┘
```
The Hub never has Firebird credentials and never opens a Firebird
connection. Each Agent has its own token and only ever reads/writes the one
Firebird database it was configured for.
## Hub server requirements
- Docker + Docker Compose v2.
- Enough disk for the Postgres volume (starts small; grows with
`classification_cache`/`jobs` history — no large binary data).
- One inbound port for the Hub API (default `8000`, or whatever your reverse
proxy forwards to it). Postgres itself does **not** need a host port —
`docker-compose.yml` intentionally does not publish `5432`; only containers
on the compose network reach it.
## Deploying the Hub
```bash
git clone <repo> && cd Projeto
cp .env.example .env
# Edit .env: set a real POSTGRES_PASSWORD, leave AGENT_DEV_TOKEN empty,
# set APP_ENV=production, BASE_URL to the public HTTPS URL.
docker compose up -d postgres fiscal-hub
curl http://localhost:8000/health # {"status": "ok"}
```
`fiscal-hub`'s container command runs `alembic upgrade head` before starting
`uvicorn`, so schema migrations apply automatically on every deploy/restart.
Both services have `restart: unless-stopped` and Docker healthchecks
(`pg_isready` for Postgres, an HTTP GET to `/health` for the Hub).
**Never** put `AGENT_DEV_TOKEN` in a production `.env` — it auto-bootstraps
an agent from a fixed shared token on every request, fine for local dev,
wrong for a pilot with real client secrets. Leave it empty; register real
Agents with `central/scripts/register_agent.py` (see below).
### Persistence
- Postgres data lives in the named volume `postgres_data`. Back it up with
`docker exec <postgres-container> pg_dump -U gti gti_fiscal > backup.sql`
(or `pg_dumpall` for the whole cluster) on whatever schedule your
operations policy requires.
- Migrations are additive-only so far (`0001`..`0006`); always run
`alembic upgrade head` (the container does this automatically) rather than
hand-editing the schema.
### HTTPS / reverse proxy
The Hub itself speaks plain HTTP. For a pilot reachable outside the Hub's
own host, put it behind a reverse proxy (nginx, Caddy, Traefik — whatever
the GTI/Proxmox environment already standardizes on) that terminates TLS and
forwards to `fiscal-hub:8000`. Nothing in the app assumes a particular
proxy; `BASE_URL` in `.env` is informational only (not currently used to
generate absolute links, but keep it accurate for anyone reading the config).
## Registering an Agent (per client installation)
Run once per Agent installation, from anywhere with access to Postgres —
easiest is inside the running Hub container:
```bash
docker exec <fiscal-hub-container> python central/scripts/register_agent.py \
--company "Client Company Name" --name "loja-01"
```
This prints `AGENT_ID`, a freshly generated `AGENT_TOKEN` (shown once — copy
it immediately), and `COMPANY_ID`. Only the token's SHA-256 hash is ever
stored in Postgres (same `token_hash` mechanism `/v1/classify` already used
— no new auth system). Each Agent gets its own token; one Agent's token
never authenticates another Agent's jobs (`agent_id` is checked on every
Job operation).
## Configuring an Agent installation
On the client machine, copy `agent/.env.example` to `agent/.env` and fill in
the values from registration plus that machine's Firebird details:
```
GTI_HUB_URL=https://hub.example.com
AGENT_ID=<from registration>
AGENT_TOKEN=<from registration>
COMPANY_ID=<from registration>
COMPANY_UF=BA
FIREBIRD_DATABASE_PATH=C:\ERP\dados\PRODUCAO.FDB
FB_USER=...
FB_PASSWORD=...
FIREBIRD_WRITES=0
POLL_INTERVAL_SECONDS=10
AGENT_VERSION=0.1.0
```
**`FIREBIRD_WRITES` must default to `0`.** See "Write safety" below before
ever setting it to `1`.
## Installing the Agent as a Windows Service
Requires `pywin32` (`pip install pywin32`, or `pip install -e ".[windows-service]"`
from `agent/`).
```powershell
# From an elevated (Administrator) terminal, in agent/service/:
python gti_agent_windows_service.py --startup=auto install
python gti_agent_windows_service.py start
# One-time, so Windows restarts it automatically after a crash:
sc.exe failure "GTIFiscalAgent" reset= 86400 actions= restart/60000/restart/60000/restart/60000
```
`gti_agent_windows_service.py` is a thin wrapper: it does not reimplement
polling, heartbeat, or the pipeline — it calls `run_agent_service.run()`
directly, stopping it via the Service Control Manager's own stop signal
instead of `SIGINT`/`SIGTERM` (which don't apply to services running with no
console). See the module's docstring for the exact commands.
### Install / configure / start / status / stop / restart / uninstall
```powershell
# Install (registers with SCM, Startup=Automatic)
python gti_agent_windows_service.py --startup=auto install
# Configure: edit agent/.env, no service restart needed until you `restart`
notepad agent\.env
# Start
python gti_agent_windows_service.py start
# Status
sc.exe query GTIFiscalAgent
# Stop
python gti_agent_windows_service.py stop
# Restart (picks up any agent/.env changes)
python gti_agent_windows_service.py restart
# Uninstall
python gti_agent_windows_service.py stop
python gti_agent_windows_service.py remove
```
### Updating the Agent's version
1. Stop the service: `python gti_agent_windows_service.py stop`
2. Replace the `agent/` code with the new version (git pull, or copy files).
3. Bump `AGENT_VERSION` in `agent/.env` if you track it per-release.
4. Start the service again: `python gti_agent_windows_service.py start`.
5. Confirm the new version shows up via `GET /v1/agents/me` on the Hub.
There is no auto-update mechanism yet (deliberately out of scope for this
pilot) — updates are a manual stop/replace/start.
### Running without installing as a service (dev/testing)
```bash
cd agent
python scripts/run_agent_service.py # runs forever, Ctrl+C to stop
python scripts/run_agent_service.py --max-iterations 5 # bounded, for scripting/tests
```
## Creating and checking work
See `docs/OPERATIONS.md` for the full `curl` examples
(`POST /v1/jobs` with `kind: REPROCESS_ALL` or `REPROCESS_PRODUCTS`,
`GET /v1/jobs/{id}` for status). Every Job is scoped to the Agent whose
token created it — no cross-Agent visibility.
## Checking Agent status
```bash
curl $GTI_HUB_URL/v1/agents/me -H "Authorization: Bearer $AGENT_TOKEN"
```
Returns `agent_id`, `company_id`, `enabled`, `last_seen_at`, `last_status`,
`schema_fingerprint`, `agent_version`, and a derived `status` of `ONLINE` or
`OFFLINE` (offline if no heartbeat arrived within
`AGENT_ONLINE_THRESHOLD_SECONDS`, default 120s — configurable on the Hub).
No dashboard yet; this is the query an operator or a future dashboard would
poll.
## Write safety: enabling FIREBIRD_WRITES for a real client
**Do not set `FIREBIRD_WRITES=1` against a real client database casually.**
The authorization model has two independent layers, both required:
1. **Path**: `AUTHORIZED_WRITE_DATABASE_PATH` in that installation's
`agent/.env` must be set to the *exact* absolute path of
`FIREBIRD_DATABASE_PATH` for that same installation. If it's unset, the
guard falls back to the developer-only disposable test database path
(`TESTESOFICIAIS.FDB`), so a real client path is refused automatically —
authorization is never inferred from the filename or the `.FDB`
extension.
2. **Schema fingerprint**: `EXPECTED_SCHEMA_FINGERPRINT` must equal the live
schema fingerprint that installation reports in its heartbeat
(`schema_fingerprint`, visible via `GET /v1/agents/me`). If the live
schema ever drifts from what was verified, writes are refused until an
operator re-verifies and updates this value.
Both checks happen before any write connection is opened
(`scripts/firebird_write_executor.py::assert_authorized_write_path` /
`assert_schema_fingerprint_authorized`), and before any product is even
read from Firebird for that job.
Even with both layers satisfied, auto-apply only ever touches
`C000025.IPI`, and only when the TIPI match is `rate_kind=NUMERIC` and
unambiguous. **No real client database has been written to during this or
any prior mission** — all writes so far were against `TESTESOFICIAIS.FDB`.
## Logs and diagnostics
- **Agent**: rotating local log file (`AGENT_LOG_PATH` in `agent/.env`,
default `agent/logs/agent.log`, 5MB × 5 backups). Contains startup,
heartbeat outcomes, job received/finished, products processed, write
counts, failures, and Hub reconnects — never secrets (`FB_PASSWORD` and
`AGENT_TOKEN` are never logged). Detailed per-product audit is in the
Agent's SQLite file (`AGENT_STATE_DB`, table `ipi_write_log`).
- **Hub**: structured `logging` output (visible via `docker compose logs
fiscal-hub`) for startup, health-check failures, heartbeats, and the full
Job lifecycle (created/claimed/completed/failed, with sanitized errors).
Coarse per-Job audit persists in Postgres (`jobs.result_summary`).
## Troubleshooting
| Symptom | Check |
|---|---|
| Agent shows `OFFLINE` | Is the service running (`sc.exe query GTIFiscalAgent`)? Check `agent/logs/agent.log` for heartbeat failures — usually `GTI_HUB_URL` unreachable or wrong `AGENT_TOKEN`. |
| Job stuck `PENDING` | Confirm the Agent that owns it is running and polling (its `agent_id` must match the Job's `agent_id` — an Agent never claims another Agent's Job). |
| Job `FAILED` | `GET /v1/jobs/{id}` for the sanitized `error`. Common causes: Firebird unreachable, `FIREBIRD_WRITES=1` misconfigured (see "Write safety"), Hub unreachable at report time (the Agent keeps the local `ipi_write_log` regardless). |
| `writes_attempted` always 0 with `FIREBIRD_WRITES=1` | Check both `AUTHORIZED_WRITE_DATABASE_PATH` and `EXPECTED_SCHEMA_FINGERPRINT` are set and correct for that installation. |
| Hub `/health` returns 503 | Postgres is down or unreachable; check `docker compose ps` and `docker compose logs postgres`. |
## Rollback / uninstalling an Agent
1. Stop and remove the Windows service (see above).
2. Delete the `agent/` installation directory on that machine (its SQLite
audit and logs are local-only; nothing to clean up centrally).
3. On the Hub, disable the Agent rather than deleting history: set
`enabled = false` for that `agents` row (no HTTP endpoint for this yet —
run it via `psql`/a short script against `DATABASE_URL`, the same access
`register_agent.py` uses). Its past Jobs and `classification_cache`
entries remain for audit.
## Rolling back the Hub
`docker compose down` does not remove the `postgres_data` volume — data
survives. To roll back to a previous Hub image/version, redeploy that
version's code and run `alembic upgrade head` again (migrations are
additive, so this is safe going forward; there is no tested downgrade path
for `0006`, so don't `alembic downgrade` in a pilot without restoring from a
backup first).