Files

T

gti d21a9d1140 Add 'fiscal/' from commit '102ba84500dbd9655a0471288c3e278652593ca9'

git-subtree-dir: fiscal
git-subtree-mainline: d95165fc8f
git-subtree-split: 102ba84500

2026-09-29 14:43:23 -03:00

6.8 KiB

Raw Permalink Blame History

GTI Fiscal Hub V1 - Mission 02 Design

Scope

Deliver one end-to-end vertical slice: a FastAPI fiscal hub backed by PostgreSQL, a versioned official NCM dataset, and a Windows-ready Python agent that reads the authorized Firebird database and submits 20 real products to the hub. The agent never writes to Firebird in this mission.

The Hub and agent are separate applications. The Hub owns fiscal knowledge; the agent owns ERP communication. The Hub never accesses a customer Firebird database and the agent never accesses central PostgreSQL.

Components

central/app is a modular FastAPI monolith with these conceptual modules:

API and bearer-token authentication.
Dataset registry and NCM importer.
Product classification: normalized description, GTIN, NCM, and CEST.
Fiscal determination: operation and company context with NCM/CEST, CFOP, CST/CSOSN, PIS/COFINS, IPI, and future IBS/CBS/cClassTrib inputs.
Deterministic classification cache.
Audit and provenance.
Provider interfaces, including RtcCalculatorProvider.

Product classification and fiscal determination remain separate modules inside the same process. This makes an existing valid NCM distinct from a fully proven tax treatment.

agent/src contains a Firebird reader, schema fingerprint gate, product fingerprinting, SQLite state/outbox, HTTP client, retry/backoff handling, and comparison/report generation.

docker-compose.yml starts postgres and fiscal-hub only. The Firebird database remains installed locally on Windows and outside Docker.

Hub Persistence

PostgreSQL stores datasets, ncm_entries, tenants, agents, classification_requests, classification_results, and classification_cache. Dataset metadata includes source URL, provider, download time, SHA-256, source validity/version data when present, record count, lifecycle status, and import status.

Agent tokens are supplied by environment. PostgreSQL stores only token hashes, each associated with one tenant and agent. Tenant identity is resolved from the token and must agree with request metadata.

The agent uses a separate SQLite database. It stores source and result fingerprints, fiscal dataset versions, timestamps, reason codes, retry data, and an outbox with this lifecycle:

DISCOVERED -> PENDING_API -> SENT -> RESULT_RECEIVED -> APPLIED

APPLIED is reserved for a future write-capable mission and is not reached in Mission 02.

NCM Dataset Lifecycle

The only canonical source in this mission is the Receita Federal / Portal Unico Siscomex JSON endpoint:

https://portalunico.siscomex.gov.br/classif/api/publico/nomenclatura/download/json

The importer downloads raw content, calculates SHA-256, parses JSON, and inspects its real schema before mapping fields. It must not assume field names before this inspection.

The lifecycle is:

DOWNLOAD -> HASH -> PENDING -> IMPORT -> VALIDATION -> READY -> ATOMIC PROMOTION -> ACTIVE

The previous active version becomes SUPERSEDED only inside the same short promotion transaction that activates the ready version. NCM entries are scoped to their dataset version, so clients observe either the complete prior dataset or the complete new dataset, never a mixture. Failed imports remain non-active, and prior versions remain queryable for rollback and audit.

Validation requires a successful HTTP response, parseable JSON, recognized schema, plausible nonzero record volume, persisted hash and provenance, and a fully successful import. A scheduled updater runs this same flow and promotes only a validated version.

Classification and Decisions

POST /v1/validate/ncm normalizes a NCM code and validates it against the active official dataset. /v1/classify and /v1/classify/batch accept a minimal product plus extensible company and operation context.

If an existing NCM is valid, the classification result is NCM_VALIDATED. This is not permission to populate or change CST, CFOP, CSOSN, rates, CEST, or any other fiscal field. Fields without official deterministic support return null with explicit reason codes.

Products without a valid NCM use deterministic normalized-text candidate ranking. Outcomes are HIGH_CONFIDENCE_CANDIDATE, AMBIGUOUS, or NOT_FOUND; description similarity never creates a definitive NCM classification.

Each response carries engine_version, dataset_versions (including ncm), and decision_version. These versions participate in central cache keys and agent incremental decisions. A product with unchanged source fields is reprocessed when relevant dataset or engine versions change.

can_auto_apply is always false in Mission 02 because no Firebird writes are implemented. NCM_VALIDATED is a classification status only.

Cache, Idempotency, and Audit

The Hub derives a request fingerprint from tenant, normalized product identity, relevant context, engine version, and active dataset versions. A matching cache entry returns an equivalent decision and does not create uncontrolled duplicate audit records. A changed tenant, context, engine, or dataset version cannot reuse a result.

The agent derives its product fingerprint from description, GTIN, NCM and relevant current fiscal fields. It avoids an API call only if this fingerprint, the prior decision versions, and result state all remain current. Interrupted or failed sends stay in the outbox for retry. If the API is offline, the agent does not affect the ERP and continues processing other queued items.

Provider Status

NCM is implemented. CEST, CFOP, CST/CSOSN, cClassTrib, IBS/CBS, and the RTC calculator are investigated and recorded with one of IMPLEMENTED, PARTIAL, DISCOVERED, or NOT_AVAILABLE. The RTC adapter implements the provider interface but returns NOT_CONFIGURED unless a concrete official integration is verified to be simple and safe within this scope. It never fabricates tax results.

End-to-End Proof

The agent reads the 20 sample products from C000025 using only SELECTs after its schema fingerprint equals the recorded REFERENCE_SCHEMA_V1 fingerprint. It sends them to the authenticated batch endpoint, persists results locally, and writes docs/MISSION_02_PRODUCT_RESULTS.json plus docs/MISSION_02_REPORT.md.

The report includes coverage and timing metrics, API request and cache counts, paid API counters fixed at zero, Firebird safety counters, dataset provenance, integration statuses, and real gaps. No BLOB, image, Firebird schema change, or Firebird DML/DDL is allowed.

Verification

Focused tests cover NCM import and lifecycle, normalization/ranking, authentication and tenant isolation, endpoints, cache/idempotency, agent payloads, read-only Firebird access, fingerprint gating, incremental outbox, and offline API behavior. The complete project test suite runs afterward. Docker health checks cover PostgreSQL and the Hub; Hub health reports failed when PostgreSQL is unavailable.

6.8 KiB Raw Permalink Blame History