Add 'fiscal/' from commit '102ba84500dbd9655a0471288c3e278652593ca9'
git-subtree-dir: fiscal git-subtree-mainline:d95165fc8fgit-subtree-split:102ba84500
This commit is contained in:
gti
@@ -0,0 +1,162 @@
|
||||
# GTI Fiscal Hub + Agent — Operations Guide
|
||||
|
||||
Living document. Update it when the architecture or operational flow changes;
|
||||
keep `docs/AI_PROJECT_STATE.md` as the short factual snapshot, this file as
|
||||
the "how do I actually run it" reference.
|
||||
|
||||
## RC1 operational notes (1.0.0-rc1)
|
||||
|
||||
- **Versions:** `release/VERSION`, `central/app/version.py` (`GTI_FISCAL_VERSION`), `GET /version`, `release/RELEASE_MANIFEST.json` (`python scripts/release_manifest.py <token>`: release, agent, engine/decision versions, ACTIVE dataset hashes, migrations head/applied). A release bump never changes fiscal semantics.
|
||||
- **Modes:** `FISCAL_MODE=DRY_RUN` makes writes technically impossible (Config.firebird_writes is false, `open_write_connection` refuses, the executor is never called) regardless of `FIREBIRD_WRITES`. `AUTO_APPLY` additionally needs the two pilot authorization layers (`AUTHORIZED_WRITE_DATABASE_PATH`, `EXPECTED_SCHEMA_FINGERPRINT`) and is NOT part of RC1 pilots.
|
||||
- **REPROCESS_ALL result** (stored by the Hub in the job's `result_summary`): counts of products and, in `coverage`, NCM/IPI/CEST/CFOP/CSOSN/ST/role metrics plus `company_context` (masked CNPJ, UF, CRT, source of truth).
|
||||
- **Company context:** `python agent/scripts/preflight_company_context.py` for the resolver alone; rules in `docs/AGENT_COMPANY_CONTEXT_RULES_V1.md`.
|
||||
- **Known follow-up (not RC1):** `dataset_versions()` runs ~11 queries per classification request and dominates the ~15 ms cache-hit latency; it can be memoized/cached later.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Firebird (client ERP, TESTESOFICIAIS.FDB in dev)
|
||||
│ read-only SELECT (always)
|
||||
▼
|
||||
Agent (agent/) Hub (central/) ── PostgreSQL
|
||||
- reads C000025 - classify NCM/CEST/TIPI/BA_ST_MVA
|
||||
- polls Hub for Jobs ────HTTP──▶ - owns Job queue + Agent registry
|
||||
- runs dry-run + guarded - never touches Firebird, never
|
||||
IPI write locally stores Firebird credentials
|
||||
- writes only to the same
|
||||
Firebird it read from (only
|
||||
when FIREBIRD_WRITES=1 and
|
||||
the path is the authorized
|
||||
disposable test database)
|
||||
```
|
||||
|
||||
The Hub never has Firebird credentials and never connects to any ERP
|
||||
database. All Firebird reads/writes happen inside the Agent process, on the
|
||||
machine where the ERP database lives.
|
||||
|
||||
## Bring up Hub + Postgres
|
||||
|
||||
```bash
|
||||
docker compose up -d postgres fiscal-hub
|
||||
curl http://localhost:8000/health
|
||||
```
|
||||
|
||||
The `fiscal-hub` container runs `alembic upgrade head` on startup, so schema
|
||||
migrations apply automatically. Check `docker compose logs fiscal-hub` if it
|
||||
doesn't come up healthy.
|
||||
|
||||
## Agent configuration
|
||||
|
||||
Copy `.env.example` to `.env` locally (never commit it — already gitignored)
|
||||
and set:
|
||||
|
||||
| Variable | Meaning |
|
||||
|---|---|
|
||||
| `GTI_HUB_URL` | Hub base URL, e.g. `http://localhost:8000` (falls back to `HUB_URL` for backward compatibility) |
|
||||
| `AGENT_TOKEN` | Bearer token identifying this Agent (falls back to `AGENT_DEV_TOKEN` in dev) |
|
||||
| `AGENT_ID` | Informational identifier for this Agent instance (the Hub derives the real identity from the token) |
|
||||
| `COMPANY_ID` / `COMPANY_UF` | Which client/company this Agent instance serves (informational, for logs/heartbeat) |
|
||||
| `FIREBIRD_DATABASE_PATH` | Absolute path to the client's `.FDB` file |
|
||||
| `FB_USER` / `FB_PASSWORD` | Firebird credentials — **local only, never sent to the Hub** |
|
||||
| `FIREBIRD_WRITES` | `0` (default, required) or `1`. See "Write safety" below. |
|
||||
| `POLL_INTERVAL_SECONDS` | How long to sleep when there's no pending Job (default `5`) |
|
||||
| `AGENT_VERSION` | Reported in heartbeats/job results |
|
||||
|
||||
## Registering an Agent
|
||||
|
||||
There's no separate "register" endpoint yet — an Agent is created the first
|
||||
time its bearer token is used against the Hub (`bootstrap` in
|
||||
`central/app/auth.py`), the same mechanism `/v1/classify` already used. Set
|
||||
`AGENT_TOKEN`/`AGENT_DEV_TOKEN` to a value only this Agent knows and the Hub
|
||||
will create the Agent record on first contact.
|
||||
|
||||
## Starting the Agent
|
||||
|
||||
```bash
|
||||
cd agent
|
||||
python scripts/run_agent_service.py
|
||||
```
|
||||
|
||||
On startup it prints one line confirming Hub and Firebird connectivity, then
|
||||
loops forever: heartbeat → poll for a Job → claim → execute → report →
|
||||
sleep or poll again. `Ctrl+C` (SIGINT) or SIGTERM stops it after the current
|
||||
iteration finishes.
|
||||
|
||||
For a bounded run (useful for scripting/tests), pass `--max-iterations N`.
|
||||
|
||||
## Creating work
|
||||
|
||||
```bash
|
||||
# Reprocess everything this Agent can see in C000025
|
||||
curl -X POST $GTI_HUB_URL/v1/jobs \
|
||||
-H "Authorization: Bearer $AGENT_TOKEN" -H "Content-Type: application/json" \
|
||||
-d '{"kind": "REPROCESS_ALL"}'
|
||||
|
||||
# Reprocess a specific list of CODIGOs
|
||||
curl -X POST $GTI_HUB_URL/v1/jobs \
|
||||
-H "Authorization: Bearer $AGENT_TOKEN" -H "Content-Type: application/json" \
|
||||
-d '{"kind": "REPROCESS_PRODUCTS", "payload": {"codigos": ["000026", "000027"]}}'
|
||||
```
|
||||
|
||||
Both return the Job in `PENDING` status. The Agent that owns that token is
|
||||
the only one that will ever see or claim it — `GET /v1/jobs/next` only
|
||||
returns `PENDING` jobs whose `agent_id` matches the caller.
|
||||
|
||||
## Checking status
|
||||
|
||||
```bash
|
||||
curl $GTI_HUB_URL/v1/jobs/<job_id> -H "Authorization: Bearer $AGENT_TOKEN"
|
||||
```
|
||||
|
||||
`status` is one of `PENDING`, `RUNNING`, `COMPLETED`, `FAILED`. A completed
|
||||
job's `result_summary` has `products_processed`, `writes_attempted`,
|
||||
`writes_succeeded`, `writes_failed`, `no_ops`, plus `agent_id` and
|
||||
`schema_fingerprint`. A failed job's `error` is a sanitized (truncated,
|
||||
no-secrets) message; the Agent keeps polling afterwards.
|
||||
|
||||
## Heartbeat
|
||||
|
||||
Every poll cycle the Agent calls `POST /v1/agents/heartbeat` with its
|
||||
status (`IDLE`/`BUSY`), `schema_fingerprint`, and `agent_version`. The Hub
|
||||
updates `agents.last_seen_at`, `last_status`, `schema_fingerprint`,
|
||||
`agent_version`. Check it's alive with:
|
||||
|
||||
```sql
|
||||
SELECT id, name, last_seen_at, last_status, agent_version FROM agents;
|
||||
```
|
||||
|
||||
## Write safety: FIREBIRD_WRITES
|
||||
|
||||
- `FIREBIRD_WRITES=0` (the required default): the Agent classifies every
|
||||
product, builds the dry-run decision, and logs what it *would* write —
|
||||
but issues no `UPDATE`. Jobs still complete normally.
|
||||
- `FIREBIRD_WRITES=1`: the Agent also auto-applies `C000025.IPI` when the
|
||||
proposal is unambiguous (`rate_kind=NUMERIC`, `write_eligible=true`).
|
||||
Before opening any write connection, it verifies the configured
|
||||
`FIREBIRD_DATABASE_PATH` is **exactly**
|
||||
`D:\PROJETOS\GTISISTEMAS\Projeto\TESTESOFICIAIS.FDB` (see
|
||||
`scripts/firebird_write_executor.py::assert_authorized_write_path`) and
|
||||
fails closed — no connection opened, no job attempted — if it isn't.
|
||||
See `CLAUDE.md`'s "Firebird write policy" for the full policy; this is
|
||||
the only database ever authorized for writes.
|
||||
- Auto-apply never touches CEST, EX_TIPI, CFOP, CST/CSOSN, PIS/COFINS, or
|
||||
MVA. Those remain measurement-only.
|
||||
|
||||
## Where logs/audits live
|
||||
|
||||
- **Central audit** (coarse, per Job): Postgres `jobs.result_summary` —
|
||||
counts only, queryable via `GET /v1/jobs/{id}`.
|
||||
- **Local audit** (detailed, per product per attempt): the Agent's SQLite
|
||||
file (`agent/agent_state.sqlite` by default, override with
|
||||
`AGENT_STATE_DB`), table `ipi_write_log` — one row per product per Job
|
||||
run, with before/after values, write status, and reason code. Table
|
||||
`product_state`/`outbox` still tracks the classify-request dedup from the
|
||||
original manual runner; `ipi_write_log` is independent of it.
|
||||
|
||||
## Running as a Windows Service (future)
|
||||
|
||||
`run_agent_service.py`'s `main()` is a plain, blocking, signal-aware loop —
|
||||
suitable to wrap with `pywin32`'s `win32serviceutil.ServiceFramework` (call
|
||||
`main()` or `run()` from `SvcDoRun`, use `SvcStop` to flip the same
|
||||
`GracefulStop` flag). No installer is built yet; this is a follow-up task,
|
||||
not part of this MVP.
|
||||
Reference in New Issue
Block a user