chore: capture semantic pipeline baseline

2026-09-21 15:52:17 -03:00
parent 9dcc263217
commit 6731eeca08
47 changed files with 3724 additions and 15 deletions
@@ -0,0 +1,55 @@
+# PRODUCT KNOWLEDGE BASE V1
+
+Global product knowledge for the GTI Fiscal Hub, seeded deterministically from the five
+Firebird databases (read-only) on 2026-09-21. PostgreSQL tables: `kb_products`,
+`kb_knowledge`, `kb_observations`, `tenant_product_context`.
+
+## Global vs tenant separation
+
+- `kb_products` / `kb_knowledge` — GLOBAL: only facts inherent to the product
+  (identity, validated NCM, candidate CEST/EX, semantic attributes).
+- `tenant_product_context` / `kb_observations` — TENANT: role, ST-withheld, operation
+  profile, ERP codes, per-company observations. NEVER promoted to global.
+
+## Identity
+
+Order: (1) valid GTIN (checksum-validated, exact match), (2) strong fingerprint
+sha256(normalized description | brand | unit | package quantity) — brand is not
+resolvable in this ERP generation (CODMARCA has no joinable name table), so V1 identity
+is description|unit, (3) semantic fingerprint candidate (future), (4) unknown.
+Fiscal/company context (CFOP, CSOSN, CST, rates, ST, tenant) never participates.
+
+## Knowledge provenance
+
+Every `kb_knowledge` row records `source` (OFFICIAL_DATASET | ERP_OBSERVATION | OPERATOR
+| AI | AI_PLUS_OFFICIAL_VALIDATION), confidence, evidence, model/prompt_version when
+AI-produced, dataset_versions and created_at/validated_at. History is never deleted.
+
+## Conflicts
+
+Same key with distinct values (e.g. one fingerprint declared with two different NCMs
+across tenants) is surfaced as `KNOWLEDGE_CONFLICT` (`conflicts=163` at seed). Conflicts
+are never resolved by majority and never feed another tenant silently.
+
+## Seed results (docs/runs/ai_kb_seed.json)
+
+| Metric | Value |
+|---|---|
+| Physical rows read (5 databases) | 35,066 |
+| Observations | 35,065 (1 duplicate-key row collapsed) |
+| Unique global products | 30,801 |
+| GTIN-identified products | 8,559 |
+| Fingerprint-only products | 22,242 |
+| validated_ncm (official dataset) | 30,833 knowledge rows |
+| ERP NCM observations (evidence only) | 212 products |
+| ERP CEST observations (evidence only) | 11,719 products |
+| Knowledge conflicts | 163 |
+| Firebird writes | 0 |
+
+Load is idempotent: a second run adds 0 products / 0 contexts.
+
+## Cache-first rule
+
+Any semantic task first resolves identity against the KB: EXACT (GTIN) or FINGERPRINT
+hit with non-conflicting validated knowledge answers without an AI call. Only KB misses
+or eligible conflicts reach the AI resolver.