6.8 KiB
GTI Fiscal Hub V1 - Mission 02 Design
Scope
Deliver one end-to-end vertical slice: a FastAPI fiscal hub backed by PostgreSQL, a versioned official NCM dataset, and a Windows-ready Python agent that reads the authorized Firebird database and submits 20 real products to the hub. The agent never writes to Firebird in this mission.
The Hub and agent are separate applications. The Hub owns fiscal knowledge; the agent owns ERP communication. The Hub never accesses a customer Firebird database and the agent never accesses central PostgreSQL.
Components
central/app is a modular FastAPI monolith with these conceptual modules:
- API and bearer-token authentication.
- Dataset registry and NCM importer.
- Product classification: normalized description, GTIN, NCM, and CEST.
- Fiscal determination: operation and company context with NCM/CEST, CFOP, CST/CSOSN, PIS/COFINS, IPI, and future IBS/CBS/cClassTrib inputs.
- Deterministic classification cache.
- Audit and provenance.
- Provider interfaces, including
RtcCalculatorProvider.
Product classification and fiscal determination remain separate modules inside the same process. This makes an existing valid NCM distinct from a fully proven tax treatment.
agent/src contains a Firebird reader, schema fingerprint gate, product
fingerprinting, SQLite state/outbox, HTTP client, retry/backoff handling, and
comparison/report generation.
docker-compose.yml starts postgres and fiscal-hub only. The Firebird
database remains installed locally on Windows and outside Docker.
Hub Persistence
PostgreSQL stores datasets, ncm_entries, tenants, agents,
classification_requests, classification_results, and
classification_cache. Dataset metadata includes source URL, provider,
download time, SHA-256, source validity/version data when present, record
count, lifecycle status, and import status.
Agent tokens are supplied by environment. PostgreSQL stores only token hashes, each associated with one tenant and agent. Tenant identity is resolved from the token and must agree with request metadata.
The agent uses a separate SQLite database. It stores source and result fingerprints, fiscal dataset versions, timestamps, reason codes, retry data, and an outbox with this lifecycle:
DISCOVERED -> PENDING_API -> SENT -> RESULT_RECEIVED -> APPLIED
APPLIED is reserved for a future write-capable mission and is not reached in
Mission 02.
NCM Dataset Lifecycle
The only canonical source in this mission is the Receita Federal / Portal Unico Siscomex JSON endpoint:
https://portalunico.siscomex.gov.br/classif/api/publico/nomenclatura/download/json
The importer downloads raw content, calculates SHA-256, parses JSON, and inspects its real schema before mapping fields. It must not assume field names before this inspection.
The lifecycle is:
DOWNLOAD -> HASH -> PENDING -> IMPORT -> VALIDATION -> READY -> ATOMIC PROMOTION -> ACTIVE
The previous active version becomes SUPERSEDED only inside the same short
promotion transaction that activates the ready version. NCM entries are scoped
to their dataset version, so clients observe either the complete prior dataset
or the complete new dataset, never a mixture. Failed imports remain
non-active, and prior versions remain queryable for rollback and audit.
Validation requires a successful HTTP response, parseable JSON, recognized schema, plausible nonzero record volume, persisted hash and provenance, and a fully successful import. A scheduled updater runs this same flow and promotes only a validated version.
Classification and Decisions
POST /v1/validate/ncm normalizes a NCM code and validates it against the
active official dataset. /v1/classify and /v1/classify/batch accept a
minimal product plus extensible company and operation context.
If an existing NCM is valid, the classification result is NCM_VALIDATED.
This is not permission to populate or change CST, CFOP, CSOSN, rates, CEST, or
any other fiscal field. Fields without official deterministic support return
null with explicit reason codes.
Products without a valid NCM use deterministic normalized-text candidate
ranking. Outcomes are HIGH_CONFIDENCE_CANDIDATE, AMBIGUOUS, or NOT_FOUND;
description similarity never creates a definitive NCM classification.
Each response carries engine_version, dataset_versions (including ncm),
and decision_version. These versions participate in central cache keys and
agent incremental decisions. A product with unchanged source fields is
reprocessed when relevant dataset or engine versions change.
can_auto_apply is always false in Mission 02 because no Firebird writes are
implemented. NCM_VALIDATED is a classification status only.
Cache, Idempotency, and Audit
The Hub derives a request fingerprint from tenant, normalized product identity, relevant context, engine version, and active dataset versions. A matching cache entry returns an equivalent decision and does not create uncontrolled duplicate audit records. A changed tenant, context, engine, or dataset version cannot reuse a result.
The agent derives its product fingerprint from description, GTIN, NCM and relevant current fiscal fields. It avoids an API call only if this fingerprint, the prior decision versions, and result state all remain current. Interrupted or failed sends stay in the outbox for retry. If the API is offline, the agent does not affect the ERP and continues processing other queued items.
Provider Status
NCM is implemented. CEST, CFOP, CST/CSOSN, cClassTrib, IBS/CBS, and the RTC
calculator are investigated and recorded with one of IMPLEMENTED, PARTIAL,
DISCOVERED, or NOT_AVAILABLE. The RTC adapter implements the provider
interface but returns NOT_CONFIGURED unless a concrete official integration
is verified to be simple and safe within this scope. It never fabricates tax
results.
End-to-End Proof
The agent reads the 20 sample products from C000025 using only SELECTs after
its schema fingerprint equals the recorded REFERENCE_SCHEMA_V1 fingerprint.
It sends them to the authenticated batch endpoint, persists results locally,
and writes docs/MISSION_02_PRODUCT_RESULTS.json plus
docs/MISSION_02_REPORT.md.
The report includes coverage and timing metrics, API request and cache counts, paid API counters fixed at zero, Firebird safety counters, dataset provenance, integration statuses, and real gaps. No BLOB, image, Firebird schema change, or Firebird DML/DDL is allowed.
Verification
Focused tests cover NCM import and lifecycle, normalization/ranking, authentication and tenant isolation, endpoints, cache/idempotency, agent payloads, read-only Firebird access, fingerprint gating, incremental outbox, and offline API behavior. The complete project test suite runs afterward. Docker health checks cover PostgreSQL and the Hub; Hub health reports failed when PostgreSQL is unavailable.