GTI_GERENCIADOR_FISCAL/docs/superpowers/plans/2026-09-22-autonomous-fiscal-completion-v1.md

Autonomous Fiscal Completion V1 Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Reduce deterministic fiscal residuals across the five allowlisted databases and produce a measured autonomy report without AI calls or human-answer inference.

Architecture: Add pure, data-driven requiredness and completeness modules first. Feed them sanitized read-only ERP evidence and conservative provenance-aware aggregation. Keep dry-run/write planning separate from guarded execution, and add provider/cache/budget primitives without enabling any external provider.

Tech Stack: Python 3.12, pytest, Firebird read-only discovery through existing firebird_reader, existing guarded executor, JSON/Markdown artifacts, SQLAlchemy/PostgreSQL models only for central AI ledger.

Global Constraints

• AI_WRITE_ALLOWED=false and FIREBIRD_AI_WRITES=0 throughout.
• No Z.ai, OpenAI, or any AI/provider network call.
• Human COMPANY/GROUP/PRODUCT decisions remain unanswered and unpersisted.
• NCM_CONFLICT rows and the 73 historical INSUFFICIENT_EVIDENCE conflicts remain blocked.
• Firebird access is read-only unless an exact allowlisted path passes every existing guard and the deterministic dry-run proves a write.
• Never infer fiscal origin, ST withholding, regime facts, or tenant policy from weak/default evidence.
• RETRIEVAL_V2 remains active; retrieval V3 remains unpromoted.
• Every new artifact reports provenance, unknowns, counts, and zero-call/write measurements.

File Map

• Create docs/FISCAL_REQUIREDNESS_MATRIX_V1.json: field/context requiredness rules.
• Create central/app/fiscal_completeness.py: pure field/product evaluation API.
• Create central/tests/test_fiscal_completeness.py: matrix and evaluator tests.
• Create docs/ERP_INBOUND_FISCAL_EVIDENCE_MAP_V1.json: discovered tables, relations, evidence capabilities, and counts.
• Create scripts/discover_inbound_fiscal_evidence.py: read-only schema/evidence discovery with sanitized output.
• Create central/app/evidence_aggregation.py: supplier/document observations and conservative consensus metrics.
• Create central/tests/test_evidence_aggregation.py: conflict and threshold tests.
• Create docs/ORIGIN_EVIDENCE_RESOLUTION_V1.json: origin statuses and provenance metrics.
• Create docs/AUTONOMOUS_FISCAL_COMPLETION_V1.json: per-database completeness/autonomy metrics.
• Create docs/LOCAL_PRODUCT_HEALTH_V2.json: updated health summary.
• Create docs/EXTERNAL_PRODUCT_DATA_PROVIDER_EVALUATION_V1.md: researched provider comparison without purchase/query.
• Create central/app/product_data_provider.py: provider protocol and cache normalization types.
• Create central/app/ai/budget_ledger.py: persistent budget decision interface and atomic pre-call guard.
• Create central/tests/test_product_data_provider.py and central/tests/test_ai_budget_ledger.py.
• Modify central/app/models.py and migrations only if the existing migration/test pattern supports the ledger/cache tables.
• Modify docs/AI_PROJECT_STATE.md and create docs/AUTONOMOUS_FISCAL_COMPLETION_V1.md only after measured outputs exist.

---

Task 1: Requiredness Matrix Contract

Files: Create docs/FISCAL_REQUIREDNESS_MATRIX_V1.json; test in central/tests/test_fiscal_completeness.py.

Interfaces: Matrix JSON has artifact, version, fields, contexts, and each field rule has classification, when, validity, source, and human_decision_required. The five classifications are exact strings: REQUIRED, CONDITIONAL, OPTIONAL_REFERENCE, NOT_APPLICABLE, ERP_NOT_USED.

• Write tests that load the matrix, require all 23 contract fields exactly once, reject unknown classifications, and verify NOT_APPLICABLE does not imply missing data.
• Run python -m pytest central/tests/test_fiscal_completeness.py -q; expect failure because the matrix is absent.
• Create the matrix from FISCAL_FIELD_CONTRACT_V1.json, ERP_OPERATION_PROFILES_V1.json, and known company context keys. Mark unresolved PIS/COFINS and ICMS operator facts conditional, IPI reference-only for non-contributors, service/input-only fields as conditional, and unsupported ERP fields as ERP_NOT_USED.
• Run the focused tests and confirm they pass.

Task 2: Completeness Evaluator

Files: Create central/app/fiscal_completeness.py; modify central/app/field_contract.py only if a shared field-name export is needed; test central/tests/test_fiscal_completeness.py.

Interfaces: Implement evaluate_field(field, value, context, matrix) -> dict and evaluate_product(product, context, matrix) -> dict. Field statuses are REQUIRED_AND_VALID, REQUIRED_MISSING, CONDITIONAL_AND_VALID, CONDITIONAL_NOT_TRIGGERED, OPTIONAL_VALID, OPTIONAL_EMPTY, NOT_APPLICABLE, ERP_NOT_SUPPORTED; product output includes FISCALLY_COMPLETE and fields.

• Add tests for required valid/missing, conditional triggered/untriggered, optional empty, not applicable empty, ERP unsupported, and invalid values.
• Run focused tests to confirm the new API fails before implementation.
• Implement pure evaluation with no database/network dependency; use existing contract validators where available and preserve explicit None/empty values.
• Add tests proving a blank NOT_APPLICABLE field does not make a product incomplete and a human-required company fact remains missing.
• Run focused tests and then python -m pytest central/tests/test_field_contract.py central/tests/test_fiscal_completeness.py -q.

Task 3: Read-Only Inbound Discovery

Files: Create scripts/discover_inbound_fiscal_evidence.py, docs/ERP_INBOUND_FISCAL_EVIDENCE_MAP_V1.json; tests in tests/test_inbound_evidence_discovery.py.

Interfaces: discover_database(path, connection_factory) -> dict must use read-only isolation and return sanitized table/column/relationship/evidence metadata. classify_evidence_source(table, columns) -> set[str] returns categories such as HEADER, ITEM, SUPPLIER, KEY, XML, FISCAL_ITEM, and UNKNOWN.

• Write fake-cursor tests proving table names are discovered from system metadata, XML/BLOB columns are detected by type/name, and no DML/DDL SQL is issued.
• Run the focused tests and confirm failure before implementation.
• Implement metadata discovery using the existing Firebird reader/connection conventions and parameter-free system SELECTs only; do not assume NF-e table names.
• Run discovery against the five explicit paths in read-only mode, record unavailable databases/evidence as UNAVAILABLE, and sanitize credentials, CNPJ, XML payloads, and product descriptions from the artifact.
• Validate the JSON artifact contains database/path policy, discovered tables, candidate relationships, XML availability, and row counts.

Task 4: Inbound Observation Extraction

Files: Modify scripts/discover_inbound_fiscal_evidence.py; create central/app/evidence_aggregation.py, central/tests/test_evidence_aggregation.py; create docs/INBOUND_SUPPLIER_OBSERVATIONS_V1.json.

Interfaces: Observation contains product identity, supplier/document identity hash, observed fiscal values, UF/date, and provenance. aggregate_observations(observations) -> dict returns observation_count, supplier_count, agreement_rate, latest_observation, and conflict_count per product/attribute.

• Add tests for unanimous multi-supplier evidence, split evidence, latest-date ordering, missing values, and supplier/document de-duplication.
• Run focused tests to verify failure before implementation.
• Implement conservative aggregation: agreement is strong only when at least two observations from at least two suppliers agree and no conflicting observation exists; never use simple majority as truth.
• Extract only available inbound facts; classify every observation as SUPPLIER_OBSERVATION and retain source table/document provenance.
• Produce counts for inbound items analyzed, XML documents available, and evidence-bearing products.

Task 5: Deterministic Origin Resolution

Files: Create central/app/origin_resolution.py, central/tests/test_origin_resolution.py, docs/ORIGIN_EVIDENCE_RESOLUTION_V1.json.

Interfaces: resolve_origin(product_id, observations, kb_facts) -> dict returns status in ORIGIN_CONFIRMED, ORIGIN_CONFLICT, ORIGIN_NEEDS_EVIDENCE, plus value, metrics, and provenance.

• Test that explicit consistent inbound origin can confirm, disagreement creates conflict, one weak observation remains pending, and CST digits never determine origin.
• Implement only the conservative threshold from Task 4; do not globalize a lot/acquisition-specific fact without compatible identity evidence.
• Reconcile against the existing 20,995-style pending origin counts without changing company decision artifacts or current conflicts.
• Record before/resolved/conflict/after counts and exact reasons.

Task 6: Product Role, ST/CEST, and Dynamic CFOP

Files: Create central/app/deterministic_resolution.py, central/tests/test_deterministic_resolution.py; update docs/FISCAL_REQUIREDNESS_MATRIX_V1.json and docs/AUTONOMOUS_FISCAL_COMPLETION_V1.json through generated outputs.

Interfaces: resolve_product_role(signals), resolve_st_status(signals), and resolve_entry_scope(observations, company_uf) return a value only with strong multi-signal evidence; otherwise return a typed pending result with reasons. Entry scope accepts only INTRA, INTER, MIXED_OR_DYNAMIC.

• Test conflicting role signals, recipe/production evidence, ST subject versus withheld, CEST corroboration, and mixed supplier UFs.
• Implement evidence stacks using group, recipe, stock, supplier, purchases, sales, production, flags, and history; no single weak flag may resolve role.
• Keep ST_WITHHELD pending when purchase evidence does not prove retention; do not copy company-specific CFOP/CST/CSOSN across tenants.
• Determine CFOP entry behavior as operational default/last operation/preferred default only if ERP evidence proves the behavior; otherwise report dynamic scope.
• Re-run deterministic dry-run planning for all five databases and classify each candidate without writing.

Task 7: Guarded Deterministic Write Reprocessing

Files: Modify only existing dry-run/write orchestration files required by tests; add tests/test_autonomous_write_gate.py; create docs/AUTONOMOUS_WRITE_AUDIT_V1.json.

Interfaces: eligible_write(candidate, guards) -> bool and reprocess_allowlisted(databases, dry_run) -> dict must expose writes_attempted, writes_verified, second_pass_writes, skipped reasons, and changed columns.

• Add tests proving non-allowlisted paths, missing provenance, human-required facts, conflicts, failed schema/type guards, rowcount mismatch, and post-write mismatch all fail closed.
• Run tests against mocked executor before changing orchestration.
• Wire only deterministic candidates that already satisfy the existing guarded executor contract; do not broaden field allowlists or AI flags.
• Execute first pass only after the dry-run artifact reports eligible changes; verify exact before/after and unchanged columns.
• Reprocess all five databases and require SECOND_PASS_WRITES=0; if any guard or verification fails, record pending and do not retry destructively.

Task 8: Completeness and Autonomy Metrics

Files: Create scripts/generate_autonomous_completion.py, central/tests/test_autonomy_metrics.py, docs/AUTONOMOUS_FISCAL_COMPLETION_V1.json, docs/LOCAL_PRODUCT_HEALTH_V2.json.

Interfaces: build_database_metrics(database_snapshot, evaluator, evidence) -> dict returns TOTAL_PRODUCTS, complete/incomplete counts, reason buckets, required instances, valid/missing counts, autonomous rate, KB reuse, and AI residual counts.

• Add fixture tests for all incompleteness buckets: MISSING_COMPANY_FACT, MISSING_PRODUCT_FACT, MISSING_EXTERNAL_PRODUCT_DATA, AI_SEMANTIC_CANDIDATE, DATA_CONFLICT, ERP_SCHEMA_LIMITATION.
• Implement metrics from evaluator outputs and measured artifacts, not estimates or prior report text.
• Generate five-bank metrics, AI_READY_BEFORE=511, measured AI_READY_AFTER, GTIN split, and autonomous/external/AI projected rates without calling AI.
• Add a new-install simulation that excludes tenant-specific answers and reports MANUAL_CONFIGURATION_REQUIRED_ON_INSTALL.

Task 9: External Provider Adapter and Evaluation

Files: Create central/app/product_data_provider.py, central/tests/test_product_data_provider.py, docs/EXTERNAL_PRODUCT_DATA_PROVIDER_EVALUATION_V1.md.

Interfaces: Define ProductDataProvider protocol with lookup_gtin(gtin) -> ProviderRecord | None; ProviderRecord contains provider/version/retrieved_at/raw_hash/normalized_data/license metadata. Define LocalBulkDatasetProvider as an offline-only implementation; no network provider is enabled.

• Test normalization, raw hashing, provider versioning, cache-key construction, and provider-unavailable behavior.
• Implement the protocol and offline cache boundary without adding a paid dependency.
• Document GS1, official/free sources, commercial API/dataset options, terms, cache/redistribution constraints, and cost scenarios for 35k/70-client, 100k, and 500k products. Mark unknown prices as NOT_VERIFIED rather than inventing them.
• Set recommendation to authorization-required unless an existing licensed local source is proven.

Task 10: Persistent AI Budget and Idempotency Controls

Files: Create central/app/ai/budget_ledger.py, central/tests/test_ai_budget_ledger.py, central/tests/test_ai_idempotency.py; modify central/app/models.py and add a migration only if the repository migration conventions support it.

Interfaces: BudgetLimits, UsageEntry, BudgetLedger.check_and_reserve(request) -> BudgetDecision, and result_cache_key(product, candidates, datasets, prompt, model) -> str.

• Test per-run, daily, monthly, request-count, and new-product limits; boundary equality is allowed, exceeding a limit returns AI_BUDGET_EXHAUSTED before provider invocation.
• Test atomic reservation semantics with two concurrent reservations against the same budget window.
• Implement ledger persistence using the existing central database abstraction; keep all credentials out of the ledger.
• Test identical product/candidate/dataset/prompt/model inputs produce a cache hit and zero second call; changed model or dataset version produces a new key.
• Run these tests entirely offline with fake providers and assert no network calls.

Task 11: Final Artifacts and Project State

Files: Create docs/AUTONOMOUS_FISCAL_COMPLETION_V1.md; modify docs/AI_PROJECT_STATE.md.

• Generate the consolidated report from measured JSON artifacts, including per-bank autonomy, requiredness, inbound reduction, KB reuse, provider candidates/cost/licensing, AI residual, human residual, writes, second pass, tests, and blockers.
• Update project state with the actual mission status, preserving AI_WRITE_ALLOWED=false, FIREBIRD_AI_WRITES=0, retrieval V2, and all conflict counts.
• Add tests that load the final JSON and assert required keys and safety invariants.
• Run python -m pytest tests -q and python -m pytest central/tests -q.
• Run git diff --check, artifact invariant validation, and tracked-secret scan.
• Commit implementation in coherent milestone commits; do not tag or push.

Final Verification

Run exactly:

python -m pytest tests -q
python -m pytest central/tests -q
git diff --check

The final report must state measured values, not expected values. A failed guard, unavailable inbound evidence, or unavailable external provider is a documented residual, never an inferred fiscal answer.