chore: capture semantic pipeline baseline

2026-09-21 15:52:17 -03:00
parent 9dcc263217
commit 6731eeca08
47 changed files with 3724 additions and 15 deletions
@@ -0,0 +1,74 @@
+# PRE-AI Local Completion V3 Design
+
+## Goal
+
+Finish and verify every local, deterministic part of the semantic completion
+product without external LLM calls, Firebird semantic writes, deployment, or
+new fiscal rules. When credentials become available, only the real benchmark
+execution should remain.
+
+## Scope and Order
+
+1. Offline AI plumbing: deterministic fake provider, batch correlation,
+   retries, budget enforcement, cache invalidation, and end-to-end dry-run.
+2. Knowledge base: canonical identity and conflict audit, conservative
+   deduplication, retrieval V3 evaluation, final candidate preparation, and
+   payload compaction.
+3. Operations: guided company/group questionnaires, rollback preview/guard,
+   idempotent local validation, and a concise health artifact.
+
+The phases are sequential. Phase 1 is rerun after Phase 2 with the final work
+items and candidate sets. No fake answer is persisted as semantic truth.
+
+## Safety Invariants
+
+- `DETERMINISTIC_GATE_CLOSED=true` remains unchanged.
+- `AI_WRITE_ALLOWED=false` and `FIREBIRD_AI_WRITES=0` are asserted by tests and
+  reports.
+- External HTTP/API calls are forbidden in offline commands and tests.
+- Firebird access remains read-only for discovery; no uncertain semantic value
+  is written to Firebird.
+- Tenant fiscal context is never promoted into global product identity.
+- Conflicts are retained and classified; automatic consolidation requires
+  exact structural evidence and equivalent normalized descriptions.
+- `MIXED_OR_DYNAMIC` never produces a single invented `CFOP_ENTRADA`.
+- Rollback changes only rows whose current value equals the recorded expected
+  post-write value.
+
+## Architecture
+
+The fake provider implements the existing semantic selection contract and is
+driven by named scenarios. A batch harness maps responses by `work_item_id`,
+validates candidate confinement, retries only transport failures, and records
+provider calls, token usage, and budget decisions. Cache keys include input,
+candidate set, model, prompt version, and all relevant dataset versions.
+
+KB consolidation remains additive: `canonical_product_id` is a stable global
+identity for equivalent observations, while tenant-specific values remain in
+tenant context. Conflict audit and candidate preparation are persisted as
+versioned JSON artifacts or Hub records; no fiscal decision is inferred.
+
+Operational tools are CLI-first and preview-first. They reuse the existing
+question and context persistence mechanisms, produce reviewable JSON/CSV, and
+are idempotent. The full validation command composes existing read-only
+scripts plus the new preparation/report steps.
+
+## Acceptance Criteria
+
+- Fake scenarios cover correct, wrong, abstain, invalid JSON, missing,
+  duplicate, out-of-candidate, timeout, 429, and 500 responses.
+- Batch sizes 5, 10, and 20 are tested with out-of-order responses and
+  partial/retry failures, always correlated by ID.
+- Budget overrun blocks before provider invocation; unknown prices remain
+  explicitly unknown.
+- Cache hit performs zero provider calls; each key component independently
+  causes a miss; dataset drift yields `NEEDS_REVALIDATION` or `CONFLICT`.
+- KB conflict audit contains all 163 baseline conflicts or an explicit
+  measured current count, with no silent merge.
+- Retrieval V3 never lowers GOLD recall@20 below 0.954 on the holdout and only
+  becomes active when the comparison artifact proves improvement or equal
+  recall with better recall@1/MRR.
+- Final E2E simulation completes GOLD -> gate -> SILVER/NOISY -> residual plan
+  with zero Firebird writes.
+- Full local validation is idempotent and emits `docs/LOCAL_PRODUCT_HEALTH_V1.json`.
+- Full test suite passes.