PRE-AI Local Completion V3 Design

Goal

Finish and verify every local, deterministic part of the semantic completion product without external LLM calls, Firebird semantic writes, deployment, or new fiscal rules. When credentials become available, only the real benchmark execution should remain.

Scope and Order

Offline AI plumbing: deterministic fake provider, batch correlation, retries, budget enforcement, cache invalidation, and end-to-end dry-run.
Knowledge base: canonical identity and conflict audit, conservative deduplication, retrieval V3 evaluation, final candidate preparation, and payload compaction.
Operations: guided company/group questionnaires, rollback preview/guard, idempotent local validation, and a concise health artifact.

The phases are sequential. Phase 1 is rerun after Phase 2 with the final work items and candidate sets. No fake answer is persisted as semantic truth.

Safety Invariants

DETERMINISTIC_GATE_CLOSED=true remains unchanged.
AI_WRITE_ALLOWED=false and FIREBIRD_AI_WRITES=0 are asserted by tests and reports.
External HTTP/API calls are forbidden in offline commands and tests.
Firebird access remains read-only for discovery; no uncertain semantic value is written to Firebird.
Tenant fiscal context is never promoted into global product identity.
Conflicts are retained and classified; automatic consolidation requires exact structural evidence and equivalent normalized descriptions.
MIXED_OR_DYNAMIC never produces a single invented CFOP_ENTRADA.
Rollback changes only rows whose current value equals the recorded expected post-write value.

Architecture

The fake provider implements the existing semantic selection contract and is driven by named scenarios. A batch harness maps responses by work_item_id, validates candidate confinement, retries only transport failures, and records provider calls, token usage, and budget decisions. Cache keys include input, candidate set, model, prompt version, and all relevant dataset versions.

KB consolidation remains additive: canonical_product_id is a stable global identity for equivalent observations, while tenant-specific values remain in tenant context. Conflict audit and candidate preparation are persisted as versioned JSON artifacts or Hub records; no fiscal decision is inferred.

Operational tools are CLI-first and preview-first. They reuse the existing question and context persistence mechanisms, produce reviewable JSON/CSV, and are idempotent. The full validation command composes existing read-only scripts plus the new preparation/report steps.

Acceptance Criteria

Fake scenarios cover correct, wrong, abstain, invalid JSON, missing, duplicate, out-of-candidate, timeout, 429, and 500 responses.
Batch sizes 5, 10, and 20 are tested with out-of-order responses and partial/retry failures, always correlated by ID.
Budget overrun blocks before provider invocation; unknown prices remain explicitly unknown.
Cache hit performs zero provider calls; each key component independently causes a miss; dataset drift yields NEEDS_REVALIDATION or CONFLICT.
KB conflict audit contains all 163 baseline conflicts or an explicit measured current count, with no silent merge.
Retrieval V3 never lowers GOLD recall@20 below 0.954 on the holdout and only becomes active when the comparison artifact proves improvement or equal recall with better recall@1/MRR.
Final E2E simulation completes GOLD -> gate -> SILVER/NOISY -> residual plan with zero Firebird writes.
Full local validation is idempotent and emits docs/LOCAL_PRODUCT_HEALTH_V1.json.
Full test suite passes.

3.6 KiB Raw Blame History