chore: capture semantic pipeline baseline

2026-09-21 15:52:17 -03:00
parent 9dcc263217
commit 6731eeca08
47 changed files with 3724 additions and 15 deletions
@@ -0,0 +1,40 @@
+{
+  "artifact": "AI_COST_PLAN_V1",
+  "unique_ai_work_items": 730,
+  "per_task_type": {
+    "EX_TIPI": {
+      "items": 112,
+      "input_tokens": 56000,
+      "output_tokens": 8960,
+      "cells_covered": 4130
+    },
+    "CEST": {
+      "items": 353,
+      "input_tokens": 176500,
+      "output_tokens": 28240,
+      "cells_covered": 2045
+    },
+    "PRODUCT_ROLE": {
+      "items": 41,
+      "input_tokens": 16400,
+      "output_tokens": 2460,
+      "cells_covered": 482
+    },
+    "INVALID_NCM": {
+      "items": 224,
+      "input_tokens": 156800,
+      "output_tokens": 26880,
+      "cells_covered": 1794
+    }
+  },
+  "estimated_input_tokens": 405700,
+  "estimated_output_tokens": 66540,
+  "estimated_cost_usd": 0.53878,
+  "ai_max_run_cost_usd": 5.0,
+  "budget_authorized": true,
+  "provider_configured": "baseline",
+  "model_configured": "baseline-deterministic-v1",
+  "ai_enabled": false,
+  "max_batch_size": 20,
+  "min_confidence": 0.6
+}
@@ -0,0 +1,66 @@
+{
+  "artifact": "AI_EVALUATION_V1",
+  "task": "INVALID_NCM candidate selection",
+  "pipeline": "candidate_generation -> provider_selection -> validator",
+  "provider": "baseline",
+  "model": "baseline-deterministic-v1",
+  "prompt_version": "ai-semantic-v1",
+  "dataset_versions": {
+    "ncm": "3abc0d2eb2501bbd0cc3f5e0ee3774d33f091ea305e8127ee020e4009878987e"
+  },
+  "k": 10,
+  "eval_total": 200,
+  "metrics": {
+    "EXACT_MATCH": 2,
+    "WRONG_SELECTION": 164,
+    "NONE_WHEN_KNOWN": 34,
+    "INVALID_OUTPUT": 0,
+    "RECALL_AT_K": 11
+  },
+  "accuracy_exact_match": 0.01,
+  "recall_at_k": 0.055,
+  "by_segment": {
+    "base_testes": {
+      "EXACT_MATCH": 0,
+      "WRONG_SELECTION": 27,
+      "NONE_WHEN_KNOWN": 13,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 2,
+      "TOTAL": 40
+    },
+    "construcao": {
+      "EXACT_MATCH": 1,
+      "WRONG_SELECTION": 39,
+      "NONE_WHEN_KNOWN": 0,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 4,
+      "TOTAL": 40
+    },
+    "mercado": {
+      "EXACT_MATCH": 1,
+      "WRONG_SELECTION": 35,
+      "NONE_WHEN_KNOWN": 4,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 2,
+      "TOTAL": 40
+    },
+    "moda": {
+      "EXACT_MATCH": 0,
+      "WRONG_SELECTION": 35,
+      "NONE_WHEN_KNOWN": 5,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 0,
+      "TOTAL": 40
+    },
+    "restaurante": {
+      "EXACT_MATCH": 0,
+      "WRONG_SELECTION": 28,
+      "NONE_WHEN_KNOWN": 12,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 3,
+      "TOTAL": 40
+    }
+  },
+  "cost_usd": 0.0,
+  "note": "baseline provider (deterministic, zero-cost): measures the pipeline floor; rerun with a real provider once credentials exist"
+}
@@ -1,6 +1,55 @@
 # AI Project State

-## Current status
+## Current status (real LLM evaluation V1 — 2026-09-21)
+
+`BLOCKED_EXTERNAL_AI_CREDENTIAL` (implementation complete). The entire real-LLM
+evaluation pipeline is built and tested offline: OpenAI-compatible provider
+(`central/app/ai/llm_provider.py`, key only from Hub env, injectable transport),
+strict per-item validator (candidate-list confinement, abstain semantics, 1:1
+work_item_id, calibrated confidence in [0,1]), progressive batch probing (20/10/5),
+post-AI validator (active-dataset membership), metrics + A/B/C error attribution +
+confidence-bucket calibration (`central/app/ai/eval_metrics.py`), permanent result
+cache (`ai_eval_cache`, migration 0014) with zero-call re-execution proof, and the
+runner `central/scripts/run_real_llm_ncm_eval.py` (gold-first; silver/noisy only
+after the gate: selection accuracy >= 0.95 AND high-confidence wrong rate <= 0.02;
+artifacts REAL_LLM_NCM_EVALUATION_V1/_ERRORS_/_COST_). The runner refuses to call
+anything without `AI_API_KEY` and exits BLOCKED_EXTERNAL_AI_CREDENTIAL with the exact
+operator action. Zero LLM calls, zero Firebird writes, AI_WRITE_ALLOWED=false.
+Operator action: set AI_API_KEY (optionally AI_BASE_URL/AI_MODEL) in the Hub .env.
+
+## Semantic candidate retrieval V2 (2026-09-21)
+
+`NCM_RETRIEVAL_V2_GATE_PASSED`. Deterministic hybrid retrieval (hierarchical official
+corpus enriched with ancestor descriptions + KB commercial->NCM neighbor bridge +
+KB-derived abbreviation lexicon, 644 entries; weights 0.3/0.7; leave-one-out discipline
+after catching self-retrieval leakage) lifted candidate recall from 7.5%@50 (V1) to
+HYBRID 92%@20 / 93.5%@50, GOLD 95.4%@20/@50, MRR 0.84 — meeting the LLM liberation gate
+(GOLD@20 ≥ 0.80, @50 ≥ 0.90). Benchmark quality audit: GOLD 65 / SILVER 118 / NOISY 17.
+GTIN audit: 8,559 valid, 251 cross-tenant matches, 90 GTIN-conflicting NCMs surfaced.
+Reprocessing the 750 work items: 4 KB_EXACT + 16 KB_FINGERPRINT + 1 high-confidence
+retrieval + 143 single-official-candidate (CEST/EX) resolved deterministically; 511
+items (4,181 cells) still need LLM selection; projected cost $0.54 → $0.44. Zero LLM
+calls, zero Firebird writes, AI_WRITE_ALLOWED remains false. Artifacts:
+docs/NCM_RETRIEVAL_EVALUATION_V2.json, docs/NCM_RETRIEVAL_ARCHITECTURE_V2.md,
+docs/GTIN_KB_AUDIT_V1.json, docs/AI_RESIDUAL_PLAN_V2.json.
+
+## AI semantic layer V1 (2026-09-21)
+
+`AI_SEMANTIC_LAYER_V1_IMPLEMENTED`. The deterministic gate was closed
+(OFFICIAL_RULE_RESOLVABLE_REMAINING=0, ERP_DATA_RESOLVABLE_REMAINING=0) and the Hub now
+owns a Product Knowledge Base plus a deduplicated AI work-item queue. entry_scope now
+admits MIXED_OR_DYNAMIC (CFOP_ENTRADA becomes operation-dependent; no CFOP invented).
+KB seeded read-only from all 5 databases: 30,801 global products (8,559 GTIN), 35,065
+observations, 163 surfaced knowledge conflicts, idempotent load. 850 residual questions
+deduped into 750 work items (20 answered by KB cache alone). Budget-guarded cost plan:
+$0.54 estimated for 730 items (authorized, $0 spent — no AI credential exists here).
+The 200-sample NCM benchmark proved the honest floor: candidate recall@10 = 5.5%
+(ERP commercial wording vs official legal wording), so AI selection over closed
+candidates is capped there — READY_FOR_AI_ASSISTED_WRITE_V2=false with candidate
+generation as the named prerequisite. AI_WRITE_ALLOWED=false in V1: AI results stay
+AI_CANDIDATE_VALIDATED, never Firebird writes. Artifacts:
+docs/PRODUCT_KNOWLEDGE_BASE_V1.md, docs/AI_SEMANTIC_LAYER_V1.md,
+docs/AI_EVALUATION_V1.json, docs/AI_COST_PLAN_V1.json, docs/AI_RESIDUAL_PLAN_V1.json.

 `FISCAL_DATA_FOUNDATION_MOSTLY_COMPLETE`. Deploy/pilot work remains paused
 per the current mission's scope. The operator manually placed the 6 official
@@ -0,0 +1,32 @@
+{
+  "artifact": "AI_RESIDUAL_PLAN_V1",
+  "raw_residual_questions": 1000,
+  "ai_work_items_raw": 850,
+  "ai_work_items_deduped": 750,
+  "by_task_type": {
+    "EX_TIPI": {
+      "items": 112,
+      "cells": 4130
+    },
+    "CEST": {
+      "items": 353,
+      "cells": 2045
+    },
+    "PRODUCT_ROLE": {
+      "items": 41,
+      "cells": 482
+    },
+    "INVALID_NCM": {
+      "items": 244,
+      "cells": 1970
+    }
+  },
+  "persisted": {
+    "created": 750,
+    "updated": 0,
+    "kb_hits": 20,
+    "conflicts": 0
+  },
+  "ai_write_allowed": false,
+  "company_decisions_remaining": 14
+}
@@ -0,0 +1,19 @@
+{
+  "artifact": "AI_RESIDUAL_PLAN_V2",
+  "work_items": 750,
+  "classification": {
+    "KB_EXACT_HIT": 4,
+    "KB_FINGERPRINT_HIT": 16,
+    "HIGH_CONFIDENCE_RETRIEVAL": 1,
+    "SINGLE_OFFICIAL_CANDIDATE": 143,
+    "LLM_SELECTION_REQUIRED": 511,
+    "UNRESOLVED": 75
+  },
+  "llm_selection_required_items": 511,
+  "llm_selection_required_cells": 4181,
+  "high_confidence_threshold": 0.75,
+  "note": "HIGH_CONFIDENCE_RETRIEVAL is a candidate, not a fiscal resolution: the official validator + operator gate still apply. AI_WRITE_ALLOWED=false.",
+  "dataset_versions": {
+    "ncm": "3abc0d2eb2501bbd0cc3f5e0ee3774d33f091ea305e8127ee020e4009878987e"
+  }
+}
@@ -0,0 +1,63 @@
+# AI SEMANTIC LAYER V1
+
+Central Hub module `central/app/ai/` + tables `ai_work_items` / `ai_runs`.
+The AI is NOT a fiscal authority, never writes Firebird, never replaces official
+datasets, and never answers company-level questions.
+
+## Contract
+
+- **Cache first**: KB exact/fingerprint hits never trigger AI (20 of 750 work items
+  were answered by the KB alone in the build pass).
+- **Dedup**: pendings are grouped by knowledge needed, not per cell — 850 residual
+  questions collapsed into 750 work items (EX_TIPI 112, CEST 353, PRODUCT_ROLE 41,
+  INVALID_NCM 244).
+- **Closed candidates only**: for every task the Hub first builds the official
+  candidate list (NCM top-K, CEST links for the NCM, TIPI EX rows). A provider may
+  only answer with a candidate code or `NONE`; `result_from_dict` rejects anything
+  else (`SemanticOutputError`). Providers never see an open "what is the NCM?" question.
+- **Structured output** (strict JSON schema): task_type, input_fingerprint,
+  selected_candidate, confidence, reason, candidate_ids_seen, model, prompt_version,
+  created_at, dataset_versions, engine_version.
+- **Provenance**: every persisted AI result is `source=AI` (or
+  `AI_PLUS_OFFICIAL_VALIDATION` after passing the official validator), with model,
+  prompt_version, dataset_versions — versioned history, never overwritten.
+- **Forbidden for AI**: ST retention (ST_WITHHELD), origin (AI_ORIGIN_AUTO_APPLY=false,
+  at most NEEDS_EVIDENCE), company profile questions (PIS/COFINS profile, entry_scope,
+  ICMS profile, store_ipi_profile, CRT, UF), CFOP invention, majority conflict resolution.
+- **Budget guard**: `BudgetGuard` refuses execution above `AI_MAX_RUN_COST_USD`;
+  no fallback to a more expensive model. Credentials live only in the Hub
+  (`AI_PROVIDER`, `AI_MODEL`, `AI_MAX_BATCH_SIZE`, `AI_MAX_RUN_COST_USD`,
+  `AI_MIN_CONFIDENCE`); the Windows Agent never sees them.
+- **AI_WRITE_ALLOWED=false in V1**: validated AI results stay
+  `AI_CANDIDATE_VALIDATED`; no Firebird write is generated from AI output.
+
+## Provider abstraction
+
+`AISemanticProvider` (select / estimate). Shipped: `BaselineProvider`
+(deterministic token-overlap top-1, zero cost) used by the evaluation harness and as
+the mandatory no-credential fallback. Real providers plug in without touching the
+fiscal engine.
+
+## Evaluation result (docs/AI_EVALUATION_V1.json, 200 samples, 5 segments)
+
+Baseline floor: EXACT_MATCH 2/200 (1.0%), NONE_WHEN_KNOWN 34, WRONG_SELECTION 164,
+INVALID_OUTPUT 0, cost $0. RECALL@10=5.5%, RECALL@50=7.5%
+(docs/runs/ai_evaluation_k50.json).
+
+**Root-cause finding**: ERP commercial descriptions ("CREME DE TRATAMENTO NOVEX
+QUERATINA 1KG") share almost no vocabulary with official legal NCM descriptions
+("Outros preparações para uso dos cabelos..."), so the deterministic candidate
+generator cannot surface the true NCM in top-K — the bottleneck is candidate
+recall, not provider selection. Since AI selection is confined to the candidate
+list, any provider would be capped near this floor.
+
+**Consequence**: READY_FOR_AI_ASSISTED_WRITE_V2=false. V2 prerequisite: semantic
+candidate generation (e.g. AI-proposed chapter/heading validated against the official
+NCM tree by prefix — still closed-world), then re-run this benchmark until accuracy
+and cost are jointly healthy.
+
+## Cost plan (docs/AI_COST_PLAN_V1.json)
+
+730 unique AI work items (20 KB hits excluded), 405,700 input / 66,540 output tokens,
+estimated $0.54 against a $5.00 budget — authorized. Actual spend $0 (baseline only;
+no AI credential exists in this environment).
@@ -0,0 +1,81 @@
+{
+  "artifact": "GTIN_KB_AUDIT_V1",
+  "gtin_valid": 8559,
+  "gtin_invalid_raw": 0,
+  "gtin_duplicated_same_product": 256,
+  "gtin_conflicting_products": 90,
+  "gtin_cross_tenant_matches": 251,
+  "conflict_examples": [
+    {
+      "gtin": "7908429102622",
+      "ncms": [
+        "40169990",
+        "87141000"
+      ]
+    },
+    {
+      "gtin": "7898324238536",
+      "ncms": [
+        "85393100",
+        "85393111"
+      ]
+    },
+    {
+      "gtin": "7898324002410",
+      "ncms": [
+        "85393100",
+        "85393111"
+      ]
+    },
+    {
+      "gtin": "4047025341110",
+      "ncms": [
+        "27101932",
+        "38190000"
+      ]
+    },
+    {
+      "gtin": "4047025341127",
+      "ncms": [
+        "27101932",
+        "38190000"
+      ]
+    },
+    {
+      "gtin": "7898227950016",
+      "ncms": [
+        "27101932",
+        "28070010"
+      ]
+    },
+    {
+      "gtin": "7891117007128",
+      "ncms": [
+        "82015000",
+        "82130000"
+      ]
+    },
+    {
+      "gtin": "7891117097921",
+      "ncms": [
+        "82015000",
+        "82130000"
+      ]
+    },
+    {
+      "gtin": "7899882310993",
+      "ncms": [
+        "38101010",
+        "85094020"
+      ]
+    },
+    {
+      "gtin": "7891114063523",
+      "ncms": [
+        "82041100",
+        "95051000"
+      ]
+    }
+  ],
+  "policy": "same GTIN across tenants is STRONG evidence for sharing global product knowledge; tenant fiscal context is never shared"
+}
@@ -0,0 +1,84 @@
+# NCM RETRIEVAL ARCHITECTURE V2
+
+Deterministic semantic candidate retrieval (zero LLM) implemented in
+`central/app/ai/retrieval.py`. Goal: surface the correct official code among candidates
+BEFORE spending on LLM selection.
+
+## Sources merged into one ranking
+
+- **A. OFFICIAL hierarchical** — every 8-digit NCM document is enriched with its
+  ancestors' official descriptions (chapter 2-digit, position 4, subposition 6) from the
+  ACTIVE dataset, so short legal texts ("Outros") carry their path vocabulary. Search is
+  two-stage: rank positions, then expand their 8-digit descendants (top-15 positions).
+- **B. KB neighbor bridge** — the Knowledge Base maps COMMERCIAL descriptions to
+  VALIDATED NCMs (conflicting products excluded). Query = commercial-token similarity;
+  evidence per candidate = similarity, observation_count, union of distinct tenants
+  (agreement reported, never a majority decision).
+- **C. HYBRID** — single ranking, weights OFFICIAL 0.3 / KB 0.7 (tuned on the V2
+  benchmark; the KB bridge carries the signal, the official component breaks ties and
+  covers products absent from the KB).
+
+## Commercial normalization (section 7)
+
+`commercial_tokens()`: NFKD/ascii/upper stems with Portuguese plural stemming, minus
+units (UN CX FD PCT KG G ML L LT PET LATA GARRAFA ...), sizes/codes (1KG, 500ML, CX 12,
+pure numbers), and TAM/COR/REF/COD/MOD + their value token. Impact measured on the
+benchmark, not assumed.
+
+## Abbreviation lexicon (section 8)
+
+`derive_abbreviation_lexicon()`: statistical, KB-derived — a 3-5 char token that is a
+strict prefix of a longer token, both occurring across distinct products sharing the
+same validated NCM (support ≥ 3). 644 abbreviations derived (e.g. REFRI→REFRIGERANTE,
+CAM→CAMISA). No LLM involved. Query stems are expanded before official retrieval.
+
+## Leave-one-out discipline
+
+Every KB/eval measurement excludes the queried product itself from the neighbor index
+(`exclude_product_id`); the first run without it showed fake 99.5% recall — self-retrieval
+leakage — and was corrected before any reporting.
+
+## Benchmark (docs/NCM_RETRIEVAL_EVALUATION_V2.json, 200 samples, seed 42)
+
+Quality audit: GOLD 65 (GTIN or cross-tenant agreement), SILVER 118, NOISY 17 (<2
+informative tokens; not pursued).
+
+| Mode | R@1 | R@5 | R@10 | R@20 | R@50 | MRR |
+|---|---|---|---|---|---|---|
+| OFFICIAL_ONLY (hierarchical+enriched) | .015 | .08 | .11 | .14 | .195 | .046 |
+| KB_ONLY (neighbors, leave-one-out) | .665 | .85 | .895 | .925 | .935 | .750 |
+| HYBRID | .635 | .855 | .90 | .92 | .935 | .733 |
+
+GOLD × HYBRID: R@1 .785, R@10 .954, **R@20 .954**, **R@50 .954**, MRR .839.
+
+**Liberation gate (section 14): PASS** — GOLD_RECALL@20 = 0.954 ≥ 0.80 and
+GOLD_RECALL@50 = 0.954 ≥ 0.90. Real-LLM selection may now be evaluated on the same
+benchmark; AI write remains forbidden.
+
+## GTIN audit (docs/GTIN_KB_AUDIT_V1.json)
+
+VALID 8,559 (0 invalid — checksum-validated at ingest) · duplicated-same-product 256 ·
+cross-tenant matches 251 (strong knowledge-sharing evidence) · conflicting NCMs on the
+same GTIN 90 (surfaced as data-quality conflicts, examples in the artifact).
+
+## Why KB_EXACT_HITS was 0 (section 11)
+
+Root cause: the V1 work-item pass resolved identity by fingerprint only — the GTIN
+lookup was never attempted — and INVALID_NCM products have no validated NCM of their own,
+so hits only arise via a shared product with a valid NCM. V2 tries GTIN first
+(KB_EXACT_HIT=4) then fingerprint (KB_FINGERPRINT_HIT=16). No identity loosening.
+
+## Work-item reprocessing (docs/AI_RESIDUAL_PLAN_V2.json)
+
+750 items → 4 KB_EXACT_HIT + 16 KB_FINGERPRINT_HIT + 1 HIGH_CONFIDENCE_RETRIEVAL
+(hybrid ≥ 0.75, official-valid, cross-tenant ≥ 2 — a candidate, NOT an auto fiscal
+resolution) + 143 SINGLE_OFFICIAL_CANDIDATE (CEST/EX with exactly one official
+candidate) + 511 LLM_SELECTION_REQUIRED (4,181 cells) + 75 UNRESOLVED.
+
+Projected LLM cost drops $0.54 → $0.44 (730 → 511 calls).
+
+## CEST / EX TIPI (section 15)
+
+Reuse the same official-candidate lists (`cest_candidates_for_ncm`,
+`ex_tipi_candidates_for_ncm`); 143 items resolved to a single official candidate with no
+AI at all. Multi-candidate items wait for the LLM evaluation pass.
@@ -0,0 +1,322 @@
+{
+  "artifact": "NCM_RETRIEVAL_EVALUATION_V2",
+  "eval_total": 200,
+  "quality_counts": {
+    "GOLD": 65,
+    "SILVER": 118,
+    "NOISY": 17
+  },
+  "lexicon_size": 644,
+  "lexicon_sample": {
+    "100M": "100MT",
+    "10A": "10AM",
+    "10M": "10MT",
+    "20A": "20AM",
+    "20M": "20MT",
+    "220V": "220VX4600W",
+    "30M": "30MT",
+    "32M": "32MM",
+    "32X25": "32X25MM",
+    "40M": "40MM",
+    "4E0V": "4E0V4",
+    "4PC": "4PCA",
+    "50M": "50MT",
+    "5X100": "5X100MM",
+    "60M": "60MT",
+    "ACES": "ACESSORIO",
+    "ACHOC": "ACHOCOLATADO",
+    "ACO": "ACOPLADO",
+    "ACOPL": "ACOPLADA",
+    "ACQUA": "ACQUAMARINE",
+    "ACR": "ACRILICA",
+    "ACRIL": "ACRILICA",
+    "ADM": "ADMISSAO",
+    "ADVAN": "ADVANC",
+    "AERO": "AEROSOL",
+    "AFRIC": "AFRICANO",
+    "ALAV": "ALAVANCA",
+    "ALAVA": "ALAVANCA",
+    "ALC": "ALCOOL",
+    "ALE": "ALEJANDRA",
+    "ALF": "ALFAIATARIA",
+    "ALFA": "ALFAIATARIA",
+    "ALI": "ALINE",
+    "ALIC": "ALICATE",
+    "ALL": "ALLELA",
+    "ALMOF": "ALMOFADADO",
+    "ALUM": "ALUMASA",
+    "ALUMI": "ALUMINIO",
+    "AMEND": "AMENDOA",
+    "AMOR": "AMORI"
+  },
+  "overall": {
+    "OFFICIAL_ONLY": {
+      "total": 200,
+      "recall_at_1": 0.015,
+      "recall_at_5": 0.08,
+      "recall_at_10": 0.11,
+      "recall_at_20": 0.14,
+      "recall_at_50": 0.195,
+      "mrr": 0.0456
+    },
+    "KB_ONLY": {
+      "total": 200,
+      "recall_at_1": 0.665,
+      "recall_at_5": 0.85,
+      "recall_at_10": 0.895,
+      "recall_at_20": 0.925,
+      "recall_at_50": 0.935,
+      "mrr": 0.7499
+    },
+    "HYBRID": {
+      "total": 200,
+      "recall_at_1": 0.635,
+      "recall_at_5": 0.855,
+      "recall_at_10": 0.9,
+      "recall_at_20": 0.92,
+      "recall_at_50": 0.935,
+      "mrr": 0.7325
+    }
+  },
+  "by_quality": {
+    "GOLD": {
+      "OFFICIAL_ONLY": {
+        "total": 65,
+        "recall_at_1": 0.0308,
+        "recall_at_5": 0.1077,
+        "recall_at_10": 0.1231,
+        "recall_at_20": 0.1538,
+        "recall_at_50": 0.1692,
+        "mrr": 0.0681
+      },
+      "KB_ONLY": {
+        "total": 65,
+        "recall_at_1": 0.7692,
+        "recall_at_5": 0.9077,
+        "recall_at_10": 0.9385,
+        "recall_at_20": 0.9538,
+        "recall_at_50": 0.9538,
+        "mrr": 0.8276
+      },
+      "HYBRID": {
+        "total": 65,
+        "recall_at_1": 0.7846,
+        "recall_at_5": 0.9231,
+        "recall_at_10": 0.9538,
+        "recall_at_20": 0.9538,
+        "recall_at_50": 0.9538,
+        "mrr": 0.8392
+      }
+    },
+    "SILVER": {
+      "OFFICIAL_ONLY": {
+        "total": 118,
+        "recall_at_1": 0.0085,
+        "recall_at_5": 0.0763,
+        "recall_at_10": 0.1186,
+        "recall_at_20": 0.1525,
+        "recall_at_50": 0.2373,
+        "mrr": 0.0398
+      },
+      "KB_ONLY": {
+        "total": 118,
+        "recall_at_1": 0.661,
+        "recall_at_5": 0.8814,
+        "recall_at_10": 0.9237,
+        "recall_at_20": 0.9661,
+        "recall_at_50": 0.9831,
+        "mrr": 0.7656
+      },
+      "HYBRID": {
+        "total": 118,
+        "recall_at_1": 0.6271,
+        "recall_at_5": 0.8898,
+        "recall_at_10": 0.9322,
+        "recall_at_20": 0.9576,
+        "recall_at_50": 0.9831,
+        "mrr": 0.7471
+      }
+    },
+    "NOISY": {
+      "OFFICIAL_ONLY": {
+        "total": 17,
+        "recall_at_1": 0.0,
+        "recall_at_5": 0.0,
+        "recall_at_10": 0.0,
+        "recall_at_20": 0.0,
+        "recall_at_50": 0.0,
+        "mrr": 0.0
+      },
+      "KB_ONLY": {
+        "total": 17,
+        "recall_at_1": 0.2941,
+        "recall_at_5": 0.4118,
+        "recall_at_10": 0.5294,
+        "recall_at_20": 0.5294,
+        "recall_at_50": 0.5294,
+        "mrr": 0.3442
+      },
+      "HYBRID": {
+        "total": 17,
+        "recall_at_1": 0.1176,
+        "recall_at_5": 0.3529,
+        "recall_at_10": 0.4706,
+        "recall_at_20": 0.5294,
+        "recall_at_50": 0.5294,
+        "mrr": 0.2235
+      }
+    }
+  },
+  "by_segment": {
+    "base_testes": {
+      "OFFICIAL_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.0,
+        "recall_at_5": 0.05,
+        "recall_at_10": 0.075,
+        "recall_at_20": 0.075,
+        "recall_at_50": 0.1,
+        "mrr": 0.0254
+      },
+      "KB_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.5,
+        "recall_at_5": 0.75,
+        "recall_at_10": 0.825,
+        "recall_at_20": 0.85,
+        "recall_at_50": 0.85,
+        "mrr": 0.6171
+      },
+      "HYBRID": {
+        "total": 40,
+        "recall_at_1": 0.375,
+        "recall_at_5": 0.7,
+        "recall_at_10": 0.775,
+        "recall_at_20": 0.85,
+        "recall_at_50": 0.85,
+        "mrr": 0.5297
+      }
+    },
+    "construcao": {
+      "OFFICIAL_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.025,
+        "recall_at_5": 0.075,
+        "recall_at_10": 0.1,
+        "recall_at_20": 0.175,
+        "recall_at_50": 0.2,
+        "mrr": 0.0578
+      },
+      "KB_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.875,
+        "recall_at_5": 0.95,
+        "recall_at_10": 0.95,
+        "recall_at_20": 0.975,
+        "recall_at_50": 0.975,
+        "mrr": 0.9106
+      },
+      "HYBRID": {
+        "total": 40,
+        "recall_at_1": 0.85,
+        "recall_at_5": 0.975,
+        "recall_at_10": 0.975,
+        "recall_at_20": 0.975,
+        "recall_at_50": 0.975,
+        "mrr": 0.9008
+      }
+    },
+    "mercado": {
+      "OFFICIAL_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.025,
+        "recall_at_5": 0.125,
+        "recall_at_10": 0.15,
+        "recall_at_20": 0.15,
+        "recall_at_50": 0.175,
+        "mrr": 0.0637
+      },
+      "KB_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.825,
+        "recall_at_5": 0.95,
+        "recall_at_10": 0.975,
+        "recall_at_20": 0.975,
+        "recall_at_50": 0.975,
+        "mrr": 0.8807
+      },
+      "HYBRID": {
+        "total": 40,
+        "recall_at_1": 0.85,
+        "recall_at_5": 0.95,
+        "recall_at_10": 0.975,
+        "recall_at_20": 0.975,
+        "recall_at_50": 0.975,
+        "mrr": 0.8932
+      }
+    },
+    "moda": {
+      "OFFICIAL_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.025,
+        "recall_at_5": 0.15,
+        "recall_at_10": 0.225,
+        "recall_at_20": 0.25,
+        "recall_at_50": 0.425,
+        "mrr": 0.0773
+      },
+      "KB_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.425,
+        "recall_at_5": 0.725,
+        "recall_at_10": 0.85,
+        "recall_at_20": 0.95,
+        "recall_at_50": 1.0,
+        "mrr": 0.5758
+      },
+      "HYBRID": {
+        "total": 40,
+        "recall_at_1": 0.475,
+        "recall_at_5": 0.775,
+        "recall_at_10": 0.9,
+        "recall_at_20": 0.925,
+        "recall_at_50": 1.0,
+        "mrr": 0.6194
+      }
+    },
+    "restaurante": {
+      "OFFICIAL_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.0,
+        "recall_at_5": 0.0,
+        "recall_at_10": 0.0,
+        "recall_at_20": 0.05,
+        "recall_at_50": 0.075,
+        "mrr": 0.0039
+      },
+      "KB_ONLY": {
+        "total": 40,
+        "recall_at_1": 0.7,
+        "recall_at_5": 0.875,
+        "recall_at_10": 0.875,
+        "recall_at_20": 0.875,
+        "recall_at_50": 0.875,
+        "mrr": 0.7654
+      },
+      "HYBRID": {
+        "total": 40,
+        "recall_at_1": 0.625,
+        "recall_at_5": 0.875,
+        "recall_at_10": 0.875,
+        "recall_at_20": 0.875,
+        "recall_at_50": 0.875,
+        "mrr": 0.7196
+      }
+    }
+  },
+  "llm_liberation_gate": {
+    "gold_recall_at_20_min": 0.8,
+    "gold_recall_at_50_preferred": 0.9
+  },
+  "cost_usd": 0.0
+}
@@ -0,0 +1,55 @@
+# PRODUCT KNOWLEDGE BASE V1
+
+Global product knowledge for the GTI Fiscal Hub, seeded deterministically from the five
+Firebird databases (read-only) on 2026-09-21. PostgreSQL tables: `kb_products`,
+`kb_knowledge`, `kb_observations`, `tenant_product_context`.
+
+## Global vs tenant separation
+
+- `kb_products` / `kb_knowledge` — GLOBAL: only facts inherent to the product
+  (identity, validated NCM, candidate CEST/EX, semantic attributes).
+- `tenant_product_context` / `kb_observations` — TENANT: role, ST-withheld, operation
+  profile, ERP codes, per-company observations. NEVER promoted to global.
+
+## Identity
+
+Order: (1) valid GTIN (checksum-validated, exact match), (2) strong fingerprint
+sha256(normalized description | brand | unit | package quantity) — brand is not
+resolvable in this ERP generation (CODMARCA has no joinable name table), so V1 identity
+is description|unit, (3) semantic fingerprint candidate (future), (4) unknown.
+Fiscal/company context (CFOP, CSOSN, CST, rates, ST, tenant) never participates.
+
+## Knowledge provenance
+
+Every `kb_knowledge` row records `source` (OFFICIAL_DATASET | ERP_OBSERVATION | OPERATOR
+| AI | AI_PLUS_OFFICIAL_VALIDATION), confidence, evidence, model/prompt_version when
+AI-produced, dataset_versions and created_at/validated_at. History is never deleted.
+
+## Conflicts
+
+Same key with distinct values (e.g. one fingerprint declared with two different NCMs
+across tenants) is surfaced as `KNOWLEDGE_CONFLICT` (`conflicts=163` at seed). Conflicts
+are never resolved by majority and never feed another tenant silently.
+
+## Seed results (docs/runs/ai_kb_seed.json)
+
+| Metric | Value |
+|---|---|
+| Physical rows read (5 databases) | 35,066 |
+| Observations | 35,065 (1 duplicate-key row collapsed) |
+| Unique global products | 30,801 |
+| GTIN-identified products | 8,559 |
+| Fingerprint-only products | 22,242 |
+| validated_ncm (official dataset) | 30,833 knowledge rows |
+| ERP NCM observations (evidence only) | 212 products |
+| ERP CEST observations (evidence only) | 11,719 products |
+| Knowledge conflicts | 163 |
+| Firebird writes | 0 |
+
+Load is idempotent: a second run adds 0 products / 0 contexts.
+
+## Cache-first rule
+
+Any semantic task first resolves identity against the KB: EXACT (GTIN) or FINGERPRINT
+hit with non-conflicting validated knowledge answers without an AI call. Only KB misses
+or eligible conflicts reach the AI resolver.
@@ -0,0 +1,40 @@
+{
+  "artifact": "AI_COST_PLAN_V1",
+  "unique_ai_work_items": 586,
+  "per_task_type": {
+    "INVALID_NCM": {
+      "items": 223,
+      "input_tokens": 156100,
+      "output_tokens": 26760,
+      "cells_covered": 1787
+    },
+    "PRODUCT_ROLE": {
+      "items": 41,
+      "input_tokens": 16400,
+      "output_tokens": 2460,
+      "cells_covered": 482
+    },
+    "EX_TIPI": {
+      "items": 20,
+      "input_tokens": 10000,
+      "output_tokens": 1600,
+      "cells_covered": 635
+    },
+    "CEST": {
+      "items": 302,
+      "input_tokens": 151000,
+      "output_tokens": 24160,
+      "cells_covered": 1793
+    }
+  },
+  "estimated_input_tokens": 333500,
+  "estimated_output_tokens": 54980,
+  "estimated_cost_usd": 0.44346,
+  "ai_max_run_cost_usd": 5.0,
+  "budget_authorized": true,
+  "provider_configured": "baseline",
+  "model_configured": "baseline-deterministic-v1",
+  "ai_enabled": false,
+  "max_batch_size": 20,
+  "min_confidence": 0.6
+}
@@ -0,0 +1,66 @@
+{
+  "artifact": "AI_EVALUATION_V1",
+  "task": "INVALID_NCM candidate selection",
+  "pipeline": "candidate_generation -> provider_selection -> validator",
+  "provider": "baseline",
+  "model": "baseline-deterministic-v1",
+  "prompt_version": "ai-semantic-v1",
+  "dataset_versions": {
+    "ncm": "3abc0d2eb2501bbd0cc3f5e0ee3774d33f091ea305e8127ee020e4009878987e"
+  },
+  "k": 50,
+  "eval_total": 200,
+  "metrics": {
+    "EXACT_MATCH": 2,
+    "WRONG_SELECTION": 164,
+    "NONE_WHEN_KNOWN": 34,
+    "INVALID_OUTPUT": 0,
+    "RECALL_AT_K": 15
+  },
+  "accuracy_exact_match": 0.01,
+  "recall_at_k": 0.075,
+  "by_segment": {
+    "base_testes": {
+      "EXACT_MATCH": 0,
+      "WRONG_SELECTION": 27,
+      "NONE_WHEN_KNOWN": 13,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 2,
+      "TOTAL": 40
+    },
+    "construcao": {
+      "EXACT_MATCH": 1,
+      "WRONG_SELECTION": 39,
+      "NONE_WHEN_KNOWN": 0,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 5,
+      "TOTAL": 40
+    },
+    "mercado": {
+      "EXACT_MATCH": 1,
+      "WRONG_SELECTION": 35,
+      "NONE_WHEN_KNOWN": 4,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 4,
+      "TOTAL": 40
+    },
+    "moda": {
+      "EXACT_MATCH": 0,
+      "WRONG_SELECTION": 35,
+      "NONE_WHEN_KNOWN": 5,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 1,
+      "TOTAL": 40
+    },
+    "restaurante": {
+      "EXACT_MATCH": 0,
+      "WRONG_SELECTION": 28,
+      "NONE_WHEN_KNOWN": 12,
+      "INVALID_OUTPUT": 0,
+      "RECALL_AT_K": 3,
+      "TOTAL": 40
+    }
+  },
+  "cost_usd": 0.0,
+  "note": "baseline provider (deterministic, zero-cost): measures the pipeline floor; rerun with a real provider once credentials exist"
+}
@@ -0,0 +1,13 @@
+{
+  "products": 30801,
+  "observations": 35065,
+  "validated_ncm": 30833,
+  "erp_ncm_observations": 212,
+  "erp_cest_observations": 11719,
+  "tenant_contexts": 0,
+  "firebird_writes": 0,
+  "gtin_products": 8559,
+  "fingerprint_products": 22242,
+  "conflicts": 163,
+  "loaded_at": "2026-09-21T15:22:11.333546+00:00"
+}
@@ -0,0 +1,187 @@
+# PRE-AI Local Completion V3 Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Complete and verify the local semantic pipeline, knowledge preparation, and operator tooling without external AI or uncertain Firebird writes.
+
+**Architecture:** Extend the existing `central.app.ai` contracts with a deterministic scenario provider and an ID-correlated batch harness. Add conservative, versioned KB/retrieval preparation artifacts and CLI-first operational commands that compose existing read-only mechanisms.
+
+**Tech Stack:** Python 3.11+, SQLAlchemy, PostgreSQL/SQLite test engines, pytest, existing Alembic models, JSON/CSV artifacts, Docker only for existing local test execution.
+
+## Global Constraints
+
+- Do not call paid/external AI APIs.
+- Keep `DETERMINISTIC_GATE_CLOSED=true`.
+- Keep `AI_WRITE_ALLOWED=false` and `FIREBIRD_AI_WRITES=0`.
+- Never use fake answers as semantic decisions.
+- Never write uncertain semantic values to Firebird.
+- Preserve tenant-specific fiscal context outside global KB identity.
+- Preserve all conflicts and historical evidence.
+- Do not add arbitrary fiscal rules.
+
+---
+
+### Task 1: Fake Provider Contract
+
+**Files:**
+- Create: `central/app/ai/fake_provider.py`
+- Modify: `central/app/ai/provider.py`
+- Test: `central/tests/test_fake_provider.py`
+
+**Interfaces:**
+- `DeterministicFakeSemanticProvider.select_batch(items, scenario=None) -> list[dict]`
+- Scenarios: `CORRECT`, `WRONG`, `ABSTAIN`, `INVALID_JSON`, `MISSING_ITEM`, `DUPLICATE_ITEM`, `OUT_OF_CANDIDATE`, `TIMEOUT`, `429`, `500`.
+- The provider returns only test responses and never persists them.
+
+- [ ] Write tests asserting every scenario and deterministic call counters.
+- [ ] Run `python -m pytest central/tests/test_fake_provider.py -q` and observe failure.
+- [ ] Implement the provider using the existing `LlmEvalItem`/validator contract.
+- [ ] Ensure failures use typed transport/contract errors and do not mutate input items.
+- [ ] Run the focused tests and confirm pass.
+
+### Task 2: ID-Correlated Batch Harness
+
+**Files:**
+- Create: `central/app/ai/batch_runner.py`
+- Test: `central/tests/test_batch_runner.py`
+
+**Interfaces:**
+- `run_batches(items, provider, batch_size, max_retries=2) -> BatchRunResult`.
+- `BatchRunResult` exposes `results_by_work_item_id`, `errors`, `provider_calls`, `retries`, and `usage`.
+
+- [ ] Add failing tests for batch sizes 5, 10, and 20, shuffled responses, missing IDs, duplicates, invalid JSON, timeout, 429, 500, and partial failure.
+- [ ] Implement correlation by `work_item_id`, never array position.
+- [ ] Retry only transport failures and preserve failed item IDs.
+- [ ] Shrink/retry only according to explicit policy; never silently drop items.
+- [ ] Run focused tests.
+
+### Task 3: Budget, Cache, and Dataset Drift Tests
+
+**Files:**
+- Modify: `central/app/ai/provider.py`
+- Modify: `central/app/ai/llm_provider.py`
+- Create: `central/app/ai/cache_policy.py`
+- Test: `central/tests/test_ai_controls_v3.py`
+
+**Interfaces:**
+- `cache_key(input_fingerprint, candidate_set_hash, model, prompt_version, dataset_versions) -> str`.
+- `classify_knowledge_freshness(record_versions, active_versions) -> VALID | NEEDS_REVALIDATION | CONFLICT`.
+
+- [ ] Test sufficient and exceeded budgets, including zero provider calls on pre-call rejection.
+- [ ] Test input/output/cached token accounting and unknown prices.
+- [ ] Test cache hit/miss for each key component with `provider_calls=0` on hit.
+- [ ] Test dataset drift classification without deleting history.
+- [ ] Implement minimal helpers and connect them to existing cache/provider paths.
+- [ ] Run focused tests.
+
+### Task 4: Offline E2E Simulation
+
+**Files:**
+- Modify: `central/scripts/run_real_llm_ncm_eval.py`
+- Create: `central/scripts/run_offline_ai_simulation.py`
+- Test: `central/tests/test_offline_ai_simulation.py`
+
+**Interfaces:**
+- `run_offline_ai_simulation(output_dir, scenario, batch_sizes=(5,10,20)) -> dict`.
+- Output explicitly marks fake results as plumbing-only and `FIREBIRD_AI_WRITES=0`.
+
+- [ ] Test GOLD -> gate -> SILVER/NOISY -> residual plan sequencing.
+- [ ] Test that fake selections never enter Firebird or KB semantic truth tables.
+- [ ] Implement the command using final existing work-item/candidate contracts.
+- [ ] Emit sanitized metrics and zero-write evidence.
+- [ ] Run the command and focused tests.
+
+### Task 5: Canonical KB and Conflict Audit
+
+**Files:**
+- Modify: `central/app/models.py`
+- Create: `central/alembic/versions/0015_kb_canonical_identity.py`
+- Create: `central/app/ai/kb_consolidation.py`
+- Create: `central/scripts/audit_kb_conflicts.py`
+- Create: `central/tests/test_kb_consolidation.py`
+- Generate: `docs/KB_CONFLICT_AUDIT_V1.json`
+
+**Interfaces:**
+- `canonical_identity(observation) -> str` using GTIN first, then conservative fingerprint fields.
+- `classify_conflict(observations) -> str` using the six requested classifications.
+- `audit_conflicts(observations) -> dict` with counts, decisions, and evidence.
+
+- [ ] Test GTIN-equivalent abbreviation merges and incompatible same-GTIN conflicts.
+- [ ] Test packaging variations remain distinct or explicitly classified.
+- [ ] Add additive canonical identity without promoting tenant fiscal fields.
+- [ ] Audit all available KB conflicts and generate the JSON artifact.
+- [ ] Run focused tests and validate artifact counts.
+
+### Task 6: Retrieval V3 Evaluation and Candidate Preparation
+
+**Files:**
+- Modify: `central/app/ai/retrieval.py`
+- Create: `central/app/ai/retrieval_v3.py`
+- Create: `central/scripts/run_retrieval_eval_v3.py`
+- Create: `central/scripts/prepare_ai_candidates.py`
+- Test: `central/tests/test_retrieval_v3.py`
+- Generate: `docs/NCM_RETRIEVAL_EVALUATION_V3.json`
+
+**Interfaces:**
+- `retrieval_v3_candidates(corpus, index, description, metadata, k=20) -> list`.
+- `prepare_candidate_set(work_item, active_datasets) -> dict` with normalized product, top-20, scores, official descriptions, hierarchy, neighbors, GTIN evidence, and input fingerprint.
+
+- [ ] Add holdout tests for BM25/TF-IDF/character/rarity/hierarchy/KB/GTIN/group evidence.
+- [ ] Compare V3 to V2 and activate only if GOLD recall@20 is at least 0.954 and no required metric regresses.
+- [ ] Prepare NCM invalid, CEST, EX_TIPI, and PRODUCT_ROLE payloads without making semantic decisions.
+- [ ] Persist versioned candidate artifacts idempotently.
+- [ ] Run retrieval evaluation and focused tests.
+
+### Task 7: Payload Optimization and Final Phase-1 Rerun
+
+**Files:**
+- Create: `central/app/ai/payloads.py`
+- Modify: `central/scripts/run_offline_ai_simulation.py`
+- Test: `central/tests/test_payloads.py`
+- Generate: `docs/AI_PAYLOAD_OPTIMIZATION_V1.json`
+
+- [ ] Test compaction removes only redundant repeated evidence and preserves candidate recall.
+- [ ] Measure before/after token estimates for all 511 selection items.
+- [ ] Rerun Phase 1 simulation against final work items and candidate sets.
+- [ ] Confirm fake outputs remain non-authoritative and zero-write.
+
+### Task 8: Guided Company and Group Questionnaires
+
+**Files:**
+- Create: `scripts/answer_company_context.py`
+- Create: `scripts/answer_group_questions.py`
+- Create: `central/app/ai/questionnaire.py`
+- Test: `central/tests/test_questionnaire.py`
+
+- [ ] Test validation, review-before-import, cancel-without-write, JSON output, and CSV output.
+- [ ] Implement one-question-at-a-time Portuguese prompts with affected products/cells/options.
+- [ ] Group questions by company, group, subgroup, NCM, question, and consequence; show examples.
+- [ ] Enforce `ENTRY_SCOPE` values and document `MIXED_OR_DYNAMIC` behavior.
+- [ ] Implement `STORE_IPI_PROFILE` as explicit choice plus preview/rollback plan only.
+
+### Task 9: Rollback Tooling and Full Local Validation
+
+**Files:**
+- Modify: `scripts/rollback_field.py`
+- Create: `scripts/rollback_run.py`
+- Create: `scripts/run_local_validation.py`
+- Test: `tests/test_rollback_run.py`
+- Test: `tests/test_local_validation.py`
+
+- [ ] Test preview fields: run_id, database, field, product, before, current, target, affected rows.
+- [ ] Enforce `current == expected_after`; emit `CONFLICT` and never overwrite drift.
+- [ ] Compose preflight, sources, company context, KB, deterministic engine, residual classification, AI-ready preparation, and reports.
+- [ ] Make repeated runs idempotent and assert no semantic Firebird writes.
+
+### Task 10: Health Report and Final Verification
+
+**Files:**
+- Create: `scripts/generate_local_health.py`
+- Generate: `docs/LOCAL_PRODUCT_HEALTH_V1.json`
+- Modify: `docs/AI_PROJECT_STATE.md`
+
+- [ ] Generate all requested health fields with measured values, not placeholders.
+- [ ] Run Phase-1 E2E again using final Phase-2 work items/candidates.
+- [ ] Run `python -m pytest tests -q` and `python -m pytest central/tests -q`.
+- [ ] Run `git diff --check` and verify no API key or token appears in tracked artifacts.
+- [ ] Report remaining blockers precisely: external Z.ai balance/credential, missing source files, unresolved human decisions, and any measurable failures.
@@ -0,0 +1,74 @@
+# PRE-AI Local Completion V3 Design
+
+## Goal
+
+Finish and verify every local, deterministic part of the semantic completion
+product without external LLM calls, Firebird semantic writes, deployment, or
+new fiscal rules. When credentials become available, only the real benchmark
+execution should remain.
+
+## Scope and Order
+
+1. Offline AI plumbing: deterministic fake provider, batch correlation,
+   retries, budget enforcement, cache invalidation, and end-to-end dry-run.
+2. Knowledge base: canonical identity and conflict audit, conservative
+   deduplication, retrieval V3 evaluation, final candidate preparation, and
+   payload compaction.
+3. Operations: guided company/group questionnaires, rollback preview/guard,
+   idempotent local validation, and a concise health artifact.
+
+The phases are sequential. Phase 1 is rerun after Phase 2 with the final work
+items and candidate sets. No fake answer is persisted as semantic truth.
+
+## Safety Invariants
+
+- `DETERMINISTIC_GATE_CLOSED=true` remains unchanged.
+- `AI_WRITE_ALLOWED=false` and `FIREBIRD_AI_WRITES=0` are asserted by tests and
+  reports.
+- External HTTP/API calls are forbidden in offline commands and tests.
+- Firebird access remains read-only for discovery; no uncertain semantic value
+  is written to Firebird.
+- Tenant fiscal context is never promoted into global product identity.
+- Conflicts are retained and classified; automatic consolidation requires
+  exact structural evidence and equivalent normalized descriptions.
+- `MIXED_OR_DYNAMIC` never produces a single invented `CFOP_ENTRADA`.
+- Rollback changes only rows whose current value equals the recorded expected
+  post-write value.
+
+## Architecture
+
+The fake provider implements the existing semantic selection contract and is
+driven by named scenarios. A batch harness maps responses by `work_item_id`,
+validates candidate confinement, retries only transport failures, and records
+provider calls, token usage, and budget decisions. Cache keys include input,
+candidate set, model, prompt version, and all relevant dataset versions.
+
+KB consolidation remains additive: `canonical_product_id` is a stable global
+identity for equivalent observations, while tenant-specific values remain in
+tenant context. Conflict audit and candidate preparation are persisted as
+versioned JSON artifacts or Hub records; no fiscal decision is inferred.
+
+Operational tools are CLI-first and preview-first. They reuse the existing
+question and context persistence mechanisms, produce reviewable JSON/CSV, and
+are idempotent. The full validation command composes existing read-only
+scripts plus the new preparation/report steps.
+
+## Acceptance Criteria
+
+- Fake scenarios cover correct, wrong, abstain, invalid JSON, missing,
+  duplicate, out-of-candidate, timeout, 429, and 500 responses.
+- Batch sizes 5, 10, and 20 are tested with out-of-order responses and
+  partial/retry failures, always correlated by ID.
+- Budget overrun blocks before provider invocation; unknown prices remain
+  explicitly unknown.
+- Cache hit performs zero provider calls; each key component independently
+  causes a miss; dataset drift yields `NEEDS_REVALIDATION` or `CONFLICT`.
+- KB conflict audit contains all 163 baseline conflicts or an explicit
+  measured current count, with no silent merge.
+- Retrieval V3 never lowers GOLD recall@20 below 0.954 on the holdout and only
+  becomes active when the comparison artifact proves improvement or equal
+  recall with better recall@1/MRR.
+- Final E2E simulation completes GOLD -> gate -> SILVER/NOISY -> residual plan
+  with zero Firebird writes.
+- Full local validation is idempotent and emits `docs/LOCAL_PRODUCT_HEALTH_V1.json`.
+- Full test suite passes.