152 lines
6.8 KiB
Markdown
152 lines
6.8 KiB
Markdown
# GTI Fiscal Hub V1 - Mission 02 Design
|
|
|
|
## Scope
|
|
|
|
Deliver one end-to-end vertical slice: a FastAPI fiscal hub backed by
|
|
PostgreSQL, a versioned official NCM dataset, and a Windows-ready Python agent
|
|
that reads the authorized Firebird database and submits 20 real products to the
|
|
hub. The agent never writes to Firebird in this mission.
|
|
|
|
The Hub and agent are separate applications. The Hub owns fiscal knowledge;
|
|
the agent owns ERP communication. The Hub never accesses a customer Firebird
|
|
database and the agent never accesses central PostgreSQL.
|
|
|
|
## Components
|
|
|
|
`central/app` is a modular FastAPI monolith with these conceptual modules:
|
|
|
|
- API and bearer-token authentication.
|
|
- Dataset registry and NCM importer.
|
|
- Product classification: normalized description, GTIN, NCM, and CEST.
|
|
- Fiscal determination: operation and company context with NCM/CEST, CFOP,
|
|
CST/CSOSN, PIS/COFINS, IPI, and future IBS/CBS/cClassTrib inputs.
|
|
- Deterministic classification cache.
|
|
- Audit and provenance.
|
|
- Provider interfaces, including `RtcCalculatorProvider`.
|
|
|
|
Product classification and fiscal determination remain separate modules inside
|
|
the same process. This makes an existing valid NCM distinct from a fully proven
|
|
tax treatment.
|
|
|
|
`agent/src` contains a Firebird reader, schema fingerprint gate, product
|
|
fingerprinting, SQLite state/outbox, HTTP client, retry/backoff handling, and
|
|
comparison/report generation.
|
|
|
|
`docker-compose.yml` starts `postgres` and `fiscal-hub` only. The Firebird
|
|
database remains installed locally on Windows and outside Docker.
|
|
|
|
## Hub Persistence
|
|
|
|
PostgreSQL stores `datasets`, `ncm_entries`, `tenants`, `agents`,
|
|
`classification_requests`, `classification_results`, and
|
|
`classification_cache`. Dataset metadata includes source URL, provider,
|
|
download time, SHA-256, source validity/version data when present, record
|
|
count, lifecycle status, and import status.
|
|
|
|
Agent tokens are supplied by environment. PostgreSQL stores only token hashes,
|
|
each associated with one tenant and agent. Tenant identity is resolved from the
|
|
token and must agree with request metadata.
|
|
|
|
The agent uses a separate SQLite database. It stores source and result
|
|
fingerprints, fiscal dataset versions, timestamps, reason codes, retry data,
|
|
and an outbox with this lifecycle:
|
|
|
|
`DISCOVERED -> PENDING_API -> SENT -> RESULT_RECEIVED -> APPLIED`
|
|
|
|
`APPLIED` is reserved for a future write-capable mission and is not reached in
|
|
Mission 02.
|
|
|
|
## NCM Dataset Lifecycle
|
|
|
|
The only canonical source in this mission is the Receita Federal / Portal
|
|
Unico Siscomex JSON endpoint:
|
|
|
|
`https://portalunico.siscomex.gov.br/classif/api/publico/nomenclatura/download/json`
|
|
|
|
The importer downloads raw content, calculates SHA-256, parses JSON, and
|
|
inspects its real schema before mapping fields. It must not assume field names
|
|
before this inspection.
|
|
|
|
The lifecycle is:
|
|
|
|
`DOWNLOAD -> HASH -> PENDING -> IMPORT -> VALIDATION -> READY -> ATOMIC PROMOTION -> ACTIVE`
|
|
|
|
The previous active version becomes `SUPERSEDED` only inside the same short
|
|
promotion transaction that activates the ready version. NCM entries are scoped
|
|
to their dataset version, so clients observe either the complete prior dataset
|
|
or the complete new dataset, never a mixture. Failed imports remain
|
|
non-active, and prior versions remain queryable for rollback and audit.
|
|
|
|
Validation requires a successful HTTP response, parseable JSON, recognized
|
|
schema, plausible nonzero record volume, persisted hash and provenance, and a
|
|
fully successful import. A scheduled updater runs this same flow and promotes
|
|
only a validated version.
|
|
|
|
## Classification and Decisions
|
|
|
|
`POST /v1/validate/ncm` normalizes a NCM code and validates it against the
|
|
active official dataset. `/v1/classify` and `/v1/classify/batch` accept a
|
|
minimal product plus extensible company and operation context.
|
|
|
|
If an existing NCM is valid, the classification result is `NCM_VALIDATED`.
|
|
This is not permission to populate or change CST, CFOP, CSOSN, rates, CEST, or
|
|
any other fiscal field. Fields without official deterministic support return
|
|
`null` with explicit reason codes.
|
|
|
|
Products without a valid NCM use deterministic normalized-text candidate
|
|
ranking. Outcomes are `HIGH_CONFIDENCE_CANDIDATE`, `AMBIGUOUS`, or `NOT_FOUND`;
|
|
description similarity never creates a definitive NCM classification.
|
|
|
|
Each response carries `engine_version`, `dataset_versions` (including `ncm`),
|
|
and `decision_version`. These versions participate in central cache keys and
|
|
agent incremental decisions. A product with unchanged source fields is
|
|
reprocessed when relevant dataset or engine versions change.
|
|
|
|
`can_auto_apply` is always false in Mission 02 because no Firebird writes are
|
|
implemented. `NCM_VALIDATED` is a classification status only.
|
|
|
|
## Cache, Idempotency, and Audit
|
|
|
|
The Hub derives a request fingerprint from tenant, normalized product identity,
|
|
relevant context, engine version, and active dataset versions. A matching cache
|
|
entry returns an equivalent decision and does not create uncontrolled duplicate
|
|
audit records. A changed tenant, context, engine, or dataset version cannot
|
|
reuse a result.
|
|
|
|
The agent derives its product fingerprint from description, GTIN, NCM and
|
|
relevant current fiscal fields. It avoids an API call only if this fingerprint,
|
|
the prior decision versions, and result state all remain current. Interrupted
|
|
or failed sends stay in the outbox for retry. If the API is offline, the agent
|
|
does not affect the ERP and continues processing other queued items.
|
|
|
|
## Provider Status
|
|
|
|
NCM is implemented. CEST, CFOP, CST/CSOSN, cClassTrib, IBS/CBS, and the RTC
|
|
calculator are investigated and recorded with one of `IMPLEMENTED`, `PARTIAL`,
|
|
`DISCOVERED`, or `NOT_AVAILABLE`. The RTC adapter implements the provider
|
|
interface but returns `NOT_CONFIGURED` unless a concrete official integration
|
|
is verified to be simple and safe within this scope. It never fabricates tax
|
|
results.
|
|
|
|
## End-to-End Proof
|
|
|
|
The agent reads the 20 sample products from `C000025` using only SELECTs after
|
|
its schema fingerprint equals the recorded `REFERENCE_SCHEMA_V1` fingerprint.
|
|
It sends them to the authenticated batch endpoint, persists results locally,
|
|
and writes `docs/MISSION_02_PRODUCT_RESULTS.json` plus
|
|
`docs/MISSION_02_REPORT.md`.
|
|
|
|
The report includes coverage and timing metrics, API request and cache counts,
|
|
paid API counters fixed at zero, Firebird safety counters, dataset provenance,
|
|
integration statuses, and real gaps. No BLOB, image, Firebird schema change,
|
|
or Firebird DML/DDL is allowed.
|
|
|
|
## Verification
|
|
|
|
Focused tests cover NCM import and lifecycle, normalization/ranking,
|
|
authentication and tenant isolation, endpoints, cache/idempotency, agent
|
|
payloads, read-only Firebird access, fingerprint gating, incremental outbox,
|
|
and offline API behavior. The complete project test suite runs afterward.
|
|
Docker health checks cover PostgreSQL and the Hub; Hub health reports failed
|
|
when PostgreSQL is unavailable.
|