Add 'fiscal/' from commit '102ba84500dbd9655a0471288c3e278652593ca9'
git-subtree-dir: fiscal git-subtree-mainline:d95165fc8fgit-subtree-split:102ba84500
This commit is contained in:
gti
@@ -0,0 +1,85 @@
|
||||
# Mission 00: Firebird Connectivity and Discovery
|
||||
|
||||
## Goal
|
||||
|
||||
Use the Firebird installation associated with the locally installed ERP as
|
||||
read-only evidence to connect only to the authorized copy
|
||||
`D:\PROJETOS\GTISISTEMAS\Projeto\TESTESOFICIAIS.FDB`, perform complete schema
|
||||
discovery, and document confirmed findings without implementing fiscal writes,
|
||||
image writes, services, or schedulers.
|
||||
|
||||
## Scope and boundaries
|
||||
|
||||
- Files outside `D:\PROJETOS\GTISISTEMAS\Projeto` are read-only.
|
||||
- Only `TESTESOFICIAIS.FDB` may be opened or queried.
|
||||
- Other database files may be listed but must not be connected to, copied, or
|
||||
processed.
|
||||
- No DML, DDL, procedure execution, generator changes, metadata changes,
|
||||
permission changes, or service lifecycle changes are allowed.
|
||||
- Credentials and secrets found in ERP configuration are evidence of location
|
||||
only and must never enter source, reports, logs, or commits.
|
||||
|
||||
## Connection strategy
|
||||
|
||||
Use the installed Firebird server and its compatible Python driver/client as
|
||||
the primary path. First inspect services, processes, executable and DLL
|
||||
versions, tools, and ERP configuration. Select the client only after its
|
||||
version and architecture are evidenced. Use an embedded/local client only as a
|
||||
documented fallback if the server path is unavailable and compatibility is
|
||||
proven. Do not install an arbitrary Firebird version.
|
||||
|
||||
The connection layer will enforce the absolute authorized database path and
|
||||
require database-level or transaction/driver-level read-only enforcement. The
|
||||
first database operation will be a minimal identity query. The sanitized
|
||||
connection description will include driver, client, architecture, host/port
|
||||
form, and target path, but no secrets.
|
||||
|
||||
## Discovery flow
|
||||
|
||||
1. Detect Firebird capabilities and execute only safe catalog SELECT queries.
|
||||
2. Collect engine version, dialect, ODS, charset, page size, database identity,
|
||||
relation counts, fields, keys, indexes, dependencies, generators, triggers,
|
||||
procedures, parameters, and compatible monitoring metadata.
|
||||
3. Identify product candidates using names, fields, keys, indexes, counts,
|
||||
relationships, types, and minimal masked scalar samples.
|
||||
4. Map product, stock, fiscal, company/branch, image/file, audit, and
|
||||
incremental concepts. Mark every mapping `CONFIRMED`, `INFERRED`, or
|
||||
`UNKNOWN` based on direct evidence.
|
||||
5. Inspect trigger and routine source only as metadata text, without invoking
|
||||
routines or collecting source/blob contents that are sensitive or
|
||||
unnecessary.
|
||||
6. Build fingerprint V1 from engine context, product structure, fiscal fields,
|
||||
relationships, image structure, and critical triggers.
|
||||
|
||||
## Safety and sanitization
|
||||
|
||||
The existing read-only SQL boundary remains mandatory. Catalog capability
|
||||
failures are recorded as unavailable or query errors rather than treated as
|
||||
negative findings. Samples are small, scalar, masked, and purpose-limited.
|
||||
Raw BLOBs, credentials, customer data, and unnecessary commercial data are
|
||||
excluded. A path guard prevents accidental access to the ERP's original
|
||||
database even if configuration reveals it.
|
||||
|
||||
## Project changes and verification
|
||||
|
||||
Changes are limited to the authorized project directory: path guard/driver
|
||||
integration, discovery tests and fixtures, fingerprint support, and the three
|
||||
current-state documents plus sanitized mission evidence. The `.FDB` remains
|
||||
ignored by Git. Verification includes focused safety/parser tests, full
|
||||
`python -m pytest tests -q`, `git diff --check`, sensitive-data review, and
|
||||
Git status/identity inspection. A single mission commit is created only when
|
||||
Git identity is already configured; otherwise report
|
||||
`GIT_IDENTITY_NOT_CONFIGURED`.
|
||||
|
||||
## Outcomes
|
||||
|
||||
`MISSION_00_COMPLETE` requires real connection, confirmed engine details,
|
||||
evidence-backed product and fiscal mapping, image/incremental/trigger review,
|
||||
fingerprint V1, green tests, zero application writes, and no external file
|
||||
modification. Any unresolved connectivity or compatibility blocker produces
|
||||
`MISSION_00_BLOCKED` with the evidence and remaining gaps documented.
|
||||
|
||||
## Explicit non-goals
|
||||
|
||||
Do not implement fiscal rules, APIs, image storage, product UPDATEs, Windows
|
||||
services, production schedulers, migrations, triggers, or database changes.
|
||||
@@ -0,0 +1,95 @@
|
||||
# Mission 00D: Structured Firebird Read-Only Boundary
|
||||
|
||||
## Goal
|
||||
|
||||
Remove arbitrary SQL from the public safety boundary, preserve legitimate
|
||||
Firebird metadata expressions internally, then connect read-only only to
|
||||
`D:\PROJETOS\GTISISTEMAS\Projeto\TESTESOFICIAIS.FDB` and complete evidence-backed
|
||||
discovery when credentials and compatibility permit.
|
||||
|
||||
## Root cause
|
||||
|
||||
The previous `ReadOnlyConnection.execute(sql)` accepted consumer-provided SQL
|
||||
and used a blacklist/allowlist parser as its execution authority. Quoted
|
||||
identifiers could hide callable names from the textual scan, allowing potential
|
||||
procedure/UDF invocation. The same incomplete callable allowlist rejected
|
||||
legitimate Firebird expressions such as `CAST` and aggregate functions. This
|
||||
is a boundary-design failure, not a missing regex case.
|
||||
|
||||
## Architecture
|
||||
|
||||
- Remove `execute(sql)` and equivalent raw execution methods from the public
|
||||
adapter consumed by collectors and mapping code.
|
||||
- Keep driver execution in a private implementation that accepts only internal
|
||||
query identifiers and internally owned static SQL templates.
|
||||
- Expose structured catalog operations for database info, relations, columns,
|
||||
primary/foreign keys, indexes, triggers, procedures, generators,
|
||||
dependencies, and monitoring metadata.
|
||||
- Expose relation sampling only after relation and column identities have been
|
||||
returned by the catalog. Validate those identities structurally, quote them
|
||||
as Firebird identifiers, and cap the integer row limit.
|
||||
- Keep transaction-level Firebird read-only enforcement and fail closed if it
|
||||
cannot be proven. Do not expose commit, rollback, DDL, DML, procedure
|
||||
execution, or generator modification.
|
||||
- `collect_metadata` must require the approved structured reader at runtime,
|
||||
not merely by type annotation.
|
||||
|
||||
## Query ownership
|
||||
|
||||
All metadata SQL belongs to the implementation and is selected by fixed query
|
||||
keys. Internal queries may use `CAST`, `COUNT`, `SUM`, `MIN`, `MAX`,
|
||||
`COALESCE`, joins, grouping, and other required Firebird syntax without going
|
||||
through the old arbitrary-SQL parser. Procedures and UDFs are queried only as
|
||||
metadata and never invoked.
|
||||
|
||||
The old parser may remain as an offline diagnostic/test utility, but it has no
|
||||
authority to decide whether runtime SQL is executable. If retained, its
|
||||
documentation must clearly state that quoted identifiers and Firebird grammar
|
||||
are not security guarantees.
|
||||
|
||||
## Sampling
|
||||
|
||||
Sampling accepts only a relation identity previously returned by catalog
|
||||
metadata, a column set previously returned for that relation, and a validated
|
||||
small integer limit. It rejects expressions, clauses, procedures, UDFs,
|
||||
fragments, arbitrary `WHERE`/`ORDER BY`, unknown relations/columns, malformed
|
||||
quoted identifiers, and alternate database paths. It returns sanitized scalar
|
||||
evidence only; BLOB content, credentials, customer data, and unnecessary
|
||||
commercial values are excluded.
|
||||
|
||||
## Verification gates
|
||||
|
||||
Before any database access, tests must prove that arbitrary SQL cannot reach
|
||||
the driver, quoted identifiers cannot reintroduce callable execution,
|
||||
procedures/UDFs cannot be invoked, legitimate internal Firebird expressions are
|
||||
supported, and arbitrary adapters are rejected. Only after all focused tests
|
||||
and the full suite pass may the installed Firebird 2.5.9 client be used.
|
||||
|
||||
The connection then targets only the authorized `.FDB`, performs a minimal
|
||||
identity query first, and uses a read-only transaction. If credentials,
|
||||
compatibility, or identity verification fail, the result remains
|
||||
`MISSION_00_BLOCKED` and no discovery query continues.
|
||||
|
||||
## Discovery and documentation
|
||||
|
||||
After a successful connection, collect capability-aware catalog metadata,
|
||||
identify the product relation using structural and masked sample evidence,
|
||||
correlate the ERP UI fields to database fields with `CONFIRMED`, `INFERRED`,
|
||||
or `UNKNOWN` classifications, inspect fiscal/image/incremental structures,
|
||||
and analyze trigger/procedure metadata without execution. Generate a real
|
||||
schema fingerprint only from sanitized evidence. Update the current-state
|
||||
documents and final report to describe the actual adapter, parser role,
|
||||
runtime protections, transaction behavior, connection facts, and remaining
|
||||
gaps.
|
||||
|
||||
## Safety invariants
|
||||
|
||||
- `AUTHORIZED_DATABASE` is the sole file opened or queried.
|
||||
- `OTHER_DATABASES_ACCESSED = 0`.
|
||||
- `APPLICATION_DDL_DML_WRITES = 0`.
|
||||
- `ERP_FILES_MODIFIED = 0`.
|
||||
- Files outside `Projeto` are read-only.
|
||||
- No credentials, raw BLOBs, full rows, or original ERP database paths enter
|
||||
project documentation.
|
||||
- No API fiscal, image API, service, scheduler, product UPDATE, or migration
|
||||
is implemented.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Mission 00E: Threat Model, FDB Provenance, and Real Discovery
|
||||
|
||||
## Goal
|
||||
|
||||
Formalize the trusted-process threat model, validate the actual `fdb` module
|
||||
provenance and Firebird client compatibility, then run read-only discovery only
|
||||
against `D:\PROJETOS\GTISISTEMAS\Projeto\TESTESOFICIAIS.FDB`.
|
||||
|
||||
## Threat model
|
||||
|
||||
`TRUSTED_PROCESS = TRUE`: only versioned application code runs in the process;
|
||||
there are no untrusted plugins, user-provided scripts, external `eval`/`exec`,
|
||||
or externally selected dynamic imports. Deliberate introspection or monkey
|
||||
patching by code that already has arbitrary Python execution is equivalent to
|
||||
application compromise and is outside this component's boundary.
|
||||
|
||||
The boundary protects against implementation mistakes, accidental unauthorized
|
||||
operations, arbitrary SQL exposed to normal callers, parameter-to-SQL injection,
|
||||
routine execution, wrong database selection, unintended writes, and dependency
|
||||
shadowing/import mistakes.
|
||||
|
||||
## API boundary
|
||||
|
||||
The public API exposes only structured catalog operations and catalog-issued
|
||||
bounded sampling. It exposes no `execute(sql)`, `query(sql)`, `raw_execute`,
|
||||
operation-registration mutator, procedure/UDF invocation, arbitrary clauses, or
|
||||
alternate database path. The registry remains module-private, immutable, and
|
||||
centrally constructed. Python internals are not treated as a cryptographic
|
||||
sandbox.
|
||||
|
||||
The old SQL parser is offline-only diagnostic/test code. It has no authority to
|
||||
approve runtime SQL. Internal Firebird query templates may use legitimate
|
||||
`CAST`, aggregates, joins, and grouping because they are selected by fixed
|
||||
operation keys owned by the project.
|
||||
|
||||
## Provenance gate
|
||||
|
||||
Before any database connection, inspect the actual `import fdb` module:
|
||||
|
||||
- `fdb.__file__` and `fdb.__package__`;
|
||||
- module version and `importlib.metadata` distribution/version/location;
|
||||
- interpreter, working directory, `sys.path`, and `PYTHONPATH`;
|
||||
- local `fdb.py`, `fdb` directory, and unexpected `.pth` shadowing candidates.
|
||||
|
||||
The imported module must correspond to the expected installed distribution and
|
||||
version, without ambiguous duplicate installations. The installed Firebird
|
||||
client/server evidence must match the selected driver compatibility. Any
|
||||
ambiguity blocks before import/use of the driver for database access.
|
||||
|
||||
## Connection and discovery
|
||||
|
||||
After the provenance and safety gates pass, connect only to the authorized FDB,
|
||||
using the installed compatible client and a Firebird read-only transaction. Run
|
||||
the minimal identity operation first and compare the engine-reported identity
|
||||
to the authorized target. Then collect catalog metadata through structured
|
||||
operations, use only catalog-issued sample identities, and inspect product,
|
||||
stock, fiscal, image, incremental, trigger, and procedure metadata without
|
||||
executing routines or reading BLOB contents.
|
||||
|
||||
Correlate the ERP UI fields `NCM`, `CFOP`, `CST ICMS`, ICMS rate, `CEST`,
|
||||
`CSOSN`, PIS/COFINS, IPI, origin, IAT, incidence, and IPPT to table, column,
|
||||
type, domain/FK, dependencies, triggers, evidence, and confidence. Classify
|
||||
each as `CONFIRMED`, `INFERRED`, or `UNKNOWN`. Generate a real fingerprint only
|
||||
from sanitized complete evidence; otherwise use `SCHEMA_UNKNOWN`.
|
||||
|
||||
## Evidence and safety
|
||||
|
||||
- Only `D:\PROJETOS\GTISISTEMAS\Projeto\TESTESOFICIAIS.FDB` may be opened.
|
||||
- External ERP files, services, registry, DLLs, and configuration are read-only.
|
||||
- `APPLICATION_DDL_DML_WRITES = 0`.
|
||||
- `OTHER_DATABASES_ACCESSED = 0`.
|
||||
- `ERP_FILES_MODIFIED = 0`.
|
||||
- No credentials, original ERP database path, raw BLOB/image, full rows, or
|
||||
unnecessary customer/commercial data enters source or reports.
|
||||
- No fiscal API, image API, Windows service, scheduler, product UPDATE, or
|
||||
migration is implemented.
|
||||
|
||||
## Verification and outcome
|
||||
|
||||
Tests cover the public API, registry immutability, SQL non-authority,
|
||||
quoted-identifier and routine cases, legitimate internal expressions, path
|
||||
allowlisting, provenance/shadowing, client compatibility, structured reader,
|
||||
sampling, fingerprint, and discovery. The final report includes provenance,
|
||||
client/engine facts, connection result, schema mappings, image/incremental/
|
||||
trigger evidence, tests, Git state, and real gaps. Missing credentials,
|
||||
ambiguous provenance, compatibility failure, identity mismatch, or incomplete
|
||||
discovery results in `MISSION_00_BLOCKED` without a database connection or
|
||||
fallback to another database.
|
||||
@@ -0,0 +1,19 @@
|
||||
# Fiscal Data Foundation V1
|
||||
|
||||
## Goal
|
||||
|
||||
Provide independently versioned, official fiscal datasets for the Hub without determining taxes for products.
|
||||
|
||||
## Dataset lifecycle
|
||||
|
||||
Each domain runs independently: official download, raw SHA-256, versioned parser, structural validation, `PENDING` import, semantic/volume validation, then `ACTIVE` promotion. A parser failure records `FAILED` and `failure_reason`; the previous `ACTIVE` version remains queryable. No secondary source can promote a canonical dataset.
|
||||
|
||||
Every version records `domain`, `source_url`, `downloaded_at`, `raw_sha256`, `parser_version`, `effective_from`, `effective_to`, `record_count`, `status`, `failure_reason`, and `raw_payload_reference` when retention is allowed.
|
||||
|
||||
## Domains
|
||||
|
||||
Existing NCM/Siscomex and CEST/CONFAZ remain intact. New isolated importers cover BA ST/MVA, CFOP, ICMS CST/CSOSN/origin, PIS/COFINS, TIPI/IPI, RTC, FCP, and monophase ICMS/ANP. Each has a normalized operational table only when queried by Hub consumers.
|
||||
|
||||
## Verification
|
||||
|
||||
Parser tests prove fail-closed behavior and independent lifecycle behavior. The final smoke reads only `ACTIVE` versions and produces the dataset matrix. This mission does not classify products or propose tax fields.
|
||||
@@ -0,0 +1,151 @@
|
||||
# GTI Fiscal Hub V1 - Mission 02 Design
|
||||
|
||||
## Scope
|
||||
|
||||
Deliver one end-to-end vertical slice: a FastAPI fiscal hub backed by
|
||||
PostgreSQL, a versioned official NCM dataset, and a Windows-ready Python agent
|
||||
that reads the authorized Firebird database and submits 20 real products to the
|
||||
hub. The agent never writes to Firebird in this mission.
|
||||
|
||||
The Hub and agent are separate applications. The Hub owns fiscal knowledge;
|
||||
the agent owns ERP communication. The Hub never accesses a customer Firebird
|
||||
database and the agent never accesses central PostgreSQL.
|
||||
|
||||
## Components
|
||||
|
||||
`central/app` is a modular FastAPI monolith with these conceptual modules:
|
||||
|
||||
- API and bearer-token authentication.
|
||||
- Dataset registry and NCM importer.
|
||||
- Product classification: normalized description, GTIN, NCM, and CEST.
|
||||
- Fiscal determination: operation and company context with NCM/CEST, CFOP,
|
||||
CST/CSOSN, PIS/COFINS, IPI, and future IBS/CBS/cClassTrib inputs.
|
||||
- Deterministic classification cache.
|
||||
- Audit and provenance.
|
||||
- Provider interfaces, including `RtcCalculatorProvider`.
|
||||
|
||||
Product classification and fiscal determination remain separate modules inside
|
||||
the same process. This makes an existing valid NCM distinct from a fully proven
|
||||
tax treatment.
|
||||
|
||||
`agent/src` contains a Firebird reader, schema fingerprint gate, product
|
||||
fingerprinting, SQLite state/outbox, HTTP client, retry/backoff handling, and
|
||||
comparison/report generation.
|
||||
|
||||
`docker-compose.yml` starts `postgres` and `fiscal-hub` only. The Firebird
|
||||
database remains installed locally on Windows and outside Docker.
|
||||
|
||||
## Hub Persistence
|
||||
|
||||
PostgreSQL stores `datasets`, `ncm_entries`, `tenants`, `agents`,
|
||||
`classification_requests`, `classification_results`, and
|
||||
`classification_cache`. Dataset metadata includes source URL, provider,
|
||||
download time, SHA-256, source validity/version data when present, record
|
||||
count, lifecycle status, and import status.
|
||||
|
||||
Agent tokens are supplied by environment. PostgreSQL stores only token hashes,
|
||||
each associated with one tenant and agent. Tenant identity is resolved from the
|
||||
token and must agree with request metadata.
|
||||
|
||||
The agent uses a separate SQLite database. It stores source and result
|
||||
fingerprints, fiscal dataset versions, timestamps, reason codes, retry data,
|
||||
and an outbox with this lifecycle:
|
||||
|
||||
`DISCOVERED -> PENDING_API -> SENT -> RESULT_RECEIVED -> APPLIED`
|
||||
|
||||
`APPLIED` is reserved for a future write-capable mission and is not reached in
|
||||
Mission 02.
|
||||
|
||||
## NCM Dataset Lifecycle
|
||||
|
||||
The only canonical source in this mission is the Receita Federal / Portal
|
||||
Unico Siscomex JSON endpoint:
|
||||
|
||||
`https://portalunico.siscomex.gov.br/classif/api/publico/nomenclatura/download/json`
|
||||
|
||||
The importer downloads raw content, calculates SHA-256, parses JSON, and
|
||||
inspects its real schema before mapping fields. It must not assume field names
|
||||
before this inspection.
|
||||
|
||||
The lifecycle is:
|
||||
|
||||
`DOWNLOAD -> HASH -> PENDING -> IMPORT -> VALIDATION -> READY -> ATOMIC PROMOTION -> ACTIVE`
|
||||
|
||||
The previous active version becomes `SUPERSEDED` only inside the same short
|
||||
promotion transaction that activates the ready version. NCM entries are scoped
|
||||
to their dataset version, so clients observe either the complete prior dataset
|
||||
or the complete new dataset, never a mixture. Failed imports remain
|
||||
non-active, and prior versions remain queryable for rollback and audit.
|
||||
|
||||
Validation requires a successful HTTP response, parseable JSON, recognized
|
||||
schema, plausible nonzero record volume, persisted hash and provenance, and a
|
||||
fully successful import. A scheduled updater runs this same flow and promotes
|
||||
only a validated version.
|
||||
|
||||
## Classification and Decisions
|
||||
|
||||
`POST /v1/validate/ncm` normalizes a NCM code and validates it against the
|
||||
active official dataset. `/v1/classify` and `/v1/classify/batch` accept a
|
||||
minimal product plus extensible company and operation context.
|
||||
|
||||
If an existing NCM is valid, the classification result is `NCM_VALIDATED`.
|
||||
This is not permission to populate or change CST, CFOP, CSOSN, rates, CEST, or
|
||||
any other fiscal field. Fields without official deterministic support return
|
||||
`null` with explicit reason codes.
|
||||
|
||||
Products without a valid NCM use deterministic normalized-text candidate
|
||||
ranking. Outcomes are `HIGH_CONFIDENCE_CANDIDATE`, `AMBIGUOUS`, or `NOT_FOUND`;
|
||||
description similarity never creates a definitive NCM classification.
|
||||
|
||||
Each response carries `engine_version`, `dataset_versions` (including `ncm`),
|
||||
and `decision_version`. These versions participate in central cache keys and
|
||||
agent incremental decisions. A product with unchanged source fields is
|
||||
reprocessed when relevant dataset or engine versions change.
|
||||
|
||||
`can_auto_apply` is always false in Mission 02 because no Firebird writes are
|
||||
implemented. `NCM_VALIDATED` is a classification status only.
|
||||
|
||||
## Cache, Idempotency, and Audit
|
||||
|
||||
The Hub derives a request fingerprint from tenant, normalized product identity,
|
||||
relevant context, engine version, and active dataset versions. A matching cache
|
||||
entry returns an equivalent decision and does not create uncontrolled duplicate
|
||||
audit records. A changed tenant, context, engine, or dataset version cannot
|
||||
reuse a result.
|
||||
|
||||
The agent derives its product fingerprint from description, GTIN, NCM and
|
||||
relevant current fiscal fields. It avoids an API call only if this fingerprint,
|
||||
the prior decision versions, and result state all remain current. Interrupted
|
||||
or failed sends stay in the outbox for retry. If the API is offline, the agent
|
||||
does not affect the ERP and continues processing other queued items.
|
||||
|
||||
## Provider Status
|
||||
|
||||
NCM is implemented. CEST, CFOP, CST/CSOSN, cClassTrib, IBS/CBS, and the RTC
|
||||
calculator are investigated and recorded with one of `IMPLEMENTED`, `PARTIAL`,
|
||||
`DISCOVERED`, or `NOT_AVAILABLE`. The RTC adapter implements the provider
|
||||
interface but returns `NOT_CONFIGURED` unless a concrete official integration
|
||||
is verified to be simple and safe within this scope. It never fabricates tax
|
||||
results.
|
||||
|
||||
## End-to-End Proof
|
||||
|
||||
The agent reads the 20 sample products from `C000025` using only SELECTs after
|
||||
its schema fingerprint equals the recorded `REFERENCE_SCHEMA_V1` fingerprint.
|
||||
It sends them to the authenticated batch endpoint, persists results locally,
|
||||
and writes `docs/MISSION_02_PRODUCT_RESULTS.json` plus
|
||||
`docs/MISSION_02_REPORT.md`.
|
||||
|
||||
The report includes coverage and timing metrics, API request and cache counts,
|
||||
paid API counters fixed at zero, Firebird safety counters, dataset provenance,
|
||||
integration statuses, and real gaps. No BLOB, image, Firebird schema change,
|
||||
or Firebird DML/DDL is allowed.
|
||||
|
||||
## Verification
|
||||
|
||||
Focused tests cover NCM import and lifecycle, normalization/ranking,
|
||||
authentication and tenant isolation, endpoints, cache/idempotency, agent
|
||||
payloads, read-only Firebird access, fingerprint gating, incremental outbox,
|
||||
and offline API behavior. The complete project test suite runs afterward.
|
||||
Docker health checks cover PostgreSQL and the Hub; Hub health reports failed
|
||||
when PostgreSQL is unavailable.
|
||||
@@ -0,0 +1,74 @@
|
||||
# PRE-AI Local Completion V3 Design
|
||||
|
||||
## Goal
|
||||
|
||||
Finish and verify every local, deterministic part of the semantic completion
|
||||
product without external LLM calls, Firebird semantic writes, deployment, or
|
||||
new fiscal rules. When credentials become available, only the real benchmark
|
||||
execution should remain.
|
||||
|
||||
## Scope and Order
|
||||
|
||||
1. Offline AI plumbing: deterministic fake provider, batch correlation,
|
||||
retries, budget enforcement, cache invalidation, and end-to-end dry-run.
|
||||
2. Knowledge base: canonical identity and conflict audit, conservative
|
||||
deduplication, retrieval V3 evaluation, final candidate preparation, and
|
||||
payload compaction.
|
||||
3. Operations: guided company/group questionnaires, rollback preview/guard,
|
||||
idempotent local validation, and a concise health artifact.
|
||||
|
||||
The phases are sequential. Phase 1 is rerun after Phase 2 with the final work
|
||||
items and candidate sets. No fake answer is persisted as semantic truth.
|
||||
|
||||
## Safety Invariants
|
||||
|
||||
- `DETERMINISTIC_GATE_CLOSED=true` remains unchanged.
|
||||
- `AI_WRITE_ALLOWED=false` and `FIREBIRD_AI_WRITES=0` are asserted by tests and
|
||||
reports.
|
||||
- External HTTP/API calls are forbidden in offline commands and tests.
|
||||
- Firebird access remains read-only for discovery; no uncertain semantic value
|
||||
is written to Firebird.
|
||||
- Tenant fiscal context is never promoted into global product identity.
|
||||
- Conflicts are retained and classified; automatic consolidation requires
|
||||
exact structural evidence and equivalent normalized descriptions.
|
||||
- `MIXED_OR_DYNAMIC` never produces a single invented `CFOP_ENTRADA`.
|
||||
- Rollback changes only rows whose current value equals the recorded expected
|
||||
post-write value.
|
||||
|
||||
## Architecture
|
||||
|
||||
The fake provider implements the existing semantic selection contract and is
|
||||
driven by named scenarios. A batch harness maps responses by `work_item_id`,
|
||||
validates candidate confinement, retries only transport failures, and records
|
||||
provider calls, token usage, and budget decisions. Cache keys include input,
|
||||
candidate set, model, prompt version, and all relevant dataset versions.
|
||||
|
||||
KB consolidation remains additive: `canonical_product_id` is a stable global
|
||||
identity for equivalent observations, while tenant-specific values remain in
|
||||
tenant context. Conflict audit and candidate preparation are persisted as
|
||||
versioned JSON artifacts or Hub records; no fiscal decision is inferred.
|
||||
|
||||
Operational tools are CLI-first and preview-first. They reuse the existing
|
||||
question and context persistence mechanisms, produce reviewable JSON/CSV, and
|
||||
are idempotent. The full validation command composes existing read-only
|
||||
scripts plus the new preparation/report steps.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Fake scenarios cover correct, wrong, abstain, invalid JSON, missing,
|
||||
duplicate, out-of-candidate, timeout, 429, and 500 responses.
|
||||
- Batch sizes 5, 10, and 20 are tested with out-of-order responses and
|
||||
partial/retry failures, always correlated by ID.
|
||||
- Budget overrun blocks before provider invocation; unknown prices remain
|
||||
explicitly unknown.
|
||||
- Cache hit performs zero provider calls; each key component independently
|
||||
causes a miss; dataset drift yields `NEEDS_REVALIDATION` or `CONFLICT`.
|
||||
- KB conflict audit contains all 163 baseline conflicts or an explicit
|
||||
measured current count, with no silent merge.
|
||||
- Retrieval V3 never lowers GOLD recall@20 below 0.954 on the holdout and only
|
||||
becomes active when the comparison artifact proves improvement or equal
|
||||
recall with better recall@1/MRR.
|
||||
- Final E2E simulation completes GOLD -> gate -> SILVER/NOISY -> residual plan
|
||||
with zero Firebird writes.
|
||||
- Full local validation is idempotent and emits `docs/LOCAL_PRODUCT_HEALTH_V1.json`.
|
||||
- Full test suite passes.
|
||||
@@ -0,0 +1,37 @@
|
||||
# PRE-ZAI Blocker Closure Design
|
||||
|
||||
## Goal
|
||||
|
||||
Close every locally verifiable blocker before the real Z.ai GOLD benchmark,
|
||||
without changing the 90 valid NCM conflicts, inferring human facts, calling
|
||||
external AI, deploying, or enabling Firebird AI writes.
|
||||
|
||||
## Design
|
||||
|
||||
The historical conflict audit joins every available seed-payload observation by
|
||||
GTIN and fingerprint, preserves descriptions/tenant/product/NCM evidence, and
|
||||
classifies each historical row as `RESOLVED_EQUIVALENT_PRODUCT`, `FALSE_MATCH`,
|
||||
`VALID_NCM_CONFLICT`, `STALE_OBSERVATION`, or `INSUFFICIENT_EVIDENCE`. Missing
|
||||
evidence remains unresolved and never selects an NCM by majority.
|
||||
|
||||
The operator session exposes only COMPANY questions first. It is guided,
|
||||
cancelable, review-first, and produces a context import artifact only after
|
||||
explicit confirmation. IPI options include real local impact counts where the
|
||||
source artifacts expose them; rollback remains a plan until the operator
|
||||
chooses it. `MIXED_OR_DYNAMIC` produces no universal CFOP.
|
||||
|
||||
Final health derives from existing artifacts and the regenerated residual plan.
|
||||
The report uses `COMPANY_CONTEXT_COMPLETE=false` until answers are actually
|
||||
persisted, and labels GROUP/PRODUCT decisions as human-pending rather than
|
||||
pipeline failures. Retrieval V2 remains active unless a complete holdout proves
|
||||
V3 improvement.
|
||||
|
||||
## Safety and Acceptance
|
||||
|
||||
- Current 90 conflicts remain `NCM_CONFLICT` unless objective evidence changes a
|
||||
specific row.
|
||||
- Historical rows without evidence are `INSUFFICIENT_EVIDENCE`.
|
||||
- All reports state `FIREBIRD_AI_WRITES=0` and `AI_WRITE_ALLOWED=false`.
|
||||
- No answer is auto-persisted by the guided session.
|
||||
- The final artifact contains no placeholder TODOs and lists only external AI
|
||||
quota/credential plus deliberate human decisions as remaining blockers.
|
||||
@@ -0,0 +1,71 @@
|
||||
# Autonomous Fiscal Completion V1 Design
|
||||
|
||||
## Goal
|
||||
|
||||
Reduce deterministic fiscal gaps across the five explicitly allowlisted local
|
||||
Firebird databases without calling an AI provider, answering human decisions,
|
||||
or treating a populated ERP field as mandatory by default.
|
||||
|
||||
## Safety Boundary
|
||||
|
||||
All discovery and evidence extraction is read-only. Deterministic writes are
|
||||
permitted only through the existing guarded executor, only for exact allowlisted
|
||||
paths and proven write targets. A write requires provenance, `CHANGE_REQUIRED`,
|
||||
`write_eligible=true`, path/company/schema/type guards, compare-before-write,
|
||||
exact rowcount, post-write equality, and a no-unplanned-columns audit. If any
|
||||
guard fails, the result remains pending. AI flags remain
|
||||
`AI_WRITE_ALLOWED=false` and `FIREBIRD_AI_WRITES=0`.
|
||||
|
||||
## Workstreams
|
||||
|
||||
### 1. Requiredness and completeness
|
||||
|
||||
Create a versioned requiredness matrix for the 23 existing fiscal contracts.
|
||||
Rules are keyed by company regime/UF and product/operation facts, and produce
|
||||
one of `REQUIRED`, `CONDITIONAL`, `OPTIONAL_REFERENCE`, `NOT_APPLICABLE`, or
|
||||
`ERP_NOT_USED`. Implement `FiscalCompletenessEvaluator` to return a status per
|
||||
field and aggregate product metrics without requiring empty `NOT_APPLICABLE`
|
||||
fields to be filled.
|
||||
|
||||
### 2. Inbound evidence
|
||||
|
||||
Perform schema discovery across all five databases without assuming table
|
||||
names. Identify headers, items, suppliers, keys, XML/BLOB/text evidence, and
|
||||
relationships. Produce a sanitized evidence map and read-only analysis counts.
|
||||
Supplier observations remain observational until conservative deterministic
|
||||
rules prove they are reusable.
|
||||
|
||||
### 3. Deterministic resolution
|
||||
|
||||
Build provenance-aware evidence aggregation for origin, product role, ST/CEST,
|
||||
and dynamic CFOP entry scope. Use supplier/document agreement metrics rather
|
||||
than majority alone. Keep conflicts, missing evidence, company/group/product
|
||||
questions, and `ST_WITHHELD` uncertainty pending. Reuse only compatible global
|
||||
KB facts; tenant-specific policy fields are never copied cross-tenant.
|
||||
|
||||
Run dry-run and completeness metrics for all five databases. Execute only
|
||||
eligible deterministic writes, then reprocess and require zero second-pass
|
||||
writes. Preserve all current NCM conflicts.
|
||||
|
||||
### 4. Providers and AI controls
|
||||
|
||||
Document external product-data provider candidates without purchasing or
|
||||
querying them. Add an abstract provider contract and permanent-cache schema for
|
||||
GTIN/provider/version/hash/normalized data. Add persistent central AI usage
|
||||
ledger and atomic pre-call limits for run, daily, monthly, requests, and new
|
||||
products. Prove idempotent cache behavior with offline tests; no provider or AI
|
||||
network call is made.
|
||||
|
||||
## Artifacts
|
||||
|
||||
The implementation must produce the requiredness matrix, inbound evidence map,
|
||||
provider evaluation, consolidated JSON/Markdown autonomy reports, and updated
|
||||
`AI_PROJECT_STATE.md`. Artifacts must report measured values and explicit
|
||||
unknowns rather than inferred fiscal answers.
|
||||
|
||||
## Verification
|
||||
|
||||
Each workstream adds focused tests. The final gate runs the complete `tests` and
|
||||
`central/tests` suites, `git diff --check`, read-only invariant checks, dry-run
|
||||
and second-pass metrics, and confirms no external AI calls, no non-allowlisted
|
||||
database access, and no semantic Firebird writes.
|
||||
@@ -0,0 +1,111 @@
|
||||
# Control Panel Operational UX Refinement V1
|
||||
|
||||
## Scope
|
||||
|
||||
Refine the existing FastAPI/Jinja2 control panel for daily GTI operations. The
|
||||
panel must answer which client/server needs attention, why, and which safe
|
||||
operator action is available. This work does not change fiscal rules, the
|
||||
Hub/Agent protocol, Firebird access, AI write controls, or deployment.
|
||||
|
||||
## Design Choices
|
||||
|
||||
The implementation keeps the existing server-rendered architecture:
|
||||
|
||||
- `control_plane.py` remains the bounded query/view-model layer.
|
||||
- A small presentation helper formats UTC timestamps using a configurable UI
|
||||
timezone, defaulting to `America/Bahia`.
|
||||
- Existing models are reused. `FiscalContextEntry` stores validated operator
|
||||
context, `Job` schedules reprocessing, and `AuditEvent` records the complete
|
||||
action context. No new write capability is introduced.
|
||||
- Jinja2 templates receive operational view models, not raw ORM rows. Technical
|
||||
IDs remain secondary detail information.
|
||||
- CSS remains project-local and JavaScript is limited to progressive enhancement;
|
||||
blocker resolution is a server-rendered form, not a `window.confirm` action.
|
||||
|
||||
## Dashboard Semantics
|
||||
|
||||
Health cards include `ONLINE`, `DEGRADED`, `OFFLINE`, `FAILED`, and `UNKNOWN`.
|
||||
The query layer exposes a reconciled total and tests enforce:
|
||||
|
||||
`agents_total == online + degraded + offline + failed + unknown`.
|
||||
|
||||
Fiscal metrics are separate:
|
||||
|
||||
- `safe_blocked_products`: distinct active products represented by open blockers.
|
||||
- `open_blockers`: active blocker rows.
|
||||
|
||||
The dashboard labels these independently and never presents one as the other.
|
||||
Attention items are a deterministic maximum of ten, ordered by `FAILED`,
|
||||
`DEGRADED`, `OFFLINE`, `UNKNOWN`, required blockers, and important version
|
||||
drift. Each item contains client, display name, hostname, main problem, age,
|
||||
and a safe detail/filter link. Failed jobs and outdated Agents are compact
|
||||
sections with empty states.
|
||||
|
||||
## Presentation
|
||||
|
||||
UTC remains the storage/API representation. The UI renders both a human short
|
||||
form (`22/09/2026 16:11`) and relative age (`há 8 min`); the full technical
|
||||
timestamp is available as a `title` attribute. The timezone comes from
|
||||
`UI_TIMEZONE`, with `America/Bahia` as the development default.
|
||||
|
||||
Agents lead with client, display name, and hostname. Technical health and fiscal
|
||||
status are separate badges. Fiscal coverage includes total, complete, safe
|
||||
blocked, failed, and percentage with a worst-coverage sort option.
|
||||
|
||||
Jobs show client, installation/display name, hostname, job ID, kind, status,
|
||||
human timestamps, duration, and error. Blockers show client/installation,
|
||||
product, field, type, deterministic operational priority, age, and links to a
|
||||
detail page.
|
||||
|
||||
## Blocker Operations
|
||||
|
||||
The blockers page has visible server-side filters for client, installation,
|
||||
field, blocker type, status, age, and product/search. Its summary reports open
|
||||
blockers, distinct affected products, distinct affected clients, and field
|
||||
breakdown (`CFOP`, `ST`, `ICMS`, `CSOSN`, `NCM`, `OTHER`).
|
||||
|
||||
Priority is operational only and deterministic:
|
||||
|
||||
- `CRITICAL`: technical/unsafe condition that prevents processing.
|
||||
- `HIGH`: required field missing for an active product or a relevant conflict.
|
||||
- `NORMAL`: safe blocker without immediate operational urgency.
|
||||
- `LOW`: optional/reference enrichment.
|
||||
|
||||
Clicking a blocker opens a detail page with evidence, first/last seen, retry
|
||||
count, related events/jobs, and resolution history. The resolution form shows
|
||||
the client, server, product, description, field, type, reason, evidence, and
|
||||
current value. It accepts a validated context key, value, scope (`PRODUCT`,
|
||||
`GROUP`, or `COMPANY`), and reason/note. Known enum keys are validated by the
|
||||
existing `context_keys` rules; arbitrary context values cannot bypass backend
|
||||
validation.
|
||||
|
||||
On save, the transaction creates/updates the validated `FiscalContextEntry`,
|
||||
creates a bounded reprocess job for the product, and records an audit event
|
||||
containing operator, timestamp, client, installation, product, blocker,
|
||||
context key/scope/value, reason, before/after values, and reprocess job ID.
|
||||
The operation remains protected by human auth, CSRF, and authorization. It
|
||||
cannot enable AI writes, Firebird writes, SQL, or shell execution.
|
||||
|
||||
## Detail Pages and Events
|
||||
|
||||
Agent detail shows client/display name/hostname first, technical/fiscal/version
|
||||
badges, human last-seen, version, company, UF, CRT, and secondary technical IDs.
|
||||
Fiscal summary is bounded and links to filtered blockers. Event rows have human
|
||||
timestamps, severity, event type, message, and related job/product links, with
|
||||
server-side severity/type/date/job/product filters and pagination.
|
||||
|
||||
## Password Policy
|
||||
|
||||
Weak local development passwords are allowed only when explicitly enabled by
|
||||
`ALLOW_WEAK_DEV_PASSWORD=1` and only when `APP_ENV` is not `production`.
|
||||
Production always requires the strong password policy. The existing local admin
|
||||
credential is not changed.
|
||||
|
||||
## Verification
|
||||
|
||||
Tests cover health reconciliation, metrics distinction, timestamps, attention
|
||||
ordering/limit, fiscal percentage, human job identity/duration, blocker filters
|
||||
and detail, resolution validation/audit/CSRF/authorization, events, and
|
||||
environment-aware password policy. Full `tests`, `central/tests`, and
|
||||
`agent/tests` regressions plus local route/runtime and bounded performance
|
||||
smokes are required before completion.
|
||||
@@ -0,0 +1,108 @@
|
||||
# GTI Fiscal Control Panel V1 Design
|
||||
|
||||
## Scope
|
||||
|
||||
Build the operational control plane and web panel in the existing FastAPI Hub.
|
||||
The implementation has two ordered blocks: a backend foundation and a
|
||||
server-rendered interface. Fiscal decision logic remains unchanged, AI writes
|
||||
remain disabled, and no remote Firebird write control is exposed.
|
||||
|
||||
## Architecture
|
||||
|
||||
The Hub remains the single application boundary. SQLAlchemy models and Alembic
|
||||
migrations store installation identity, operational snapshots, structured
|
||||
events, human administrators, sessions, and audit entries. FastAPI exposes two
|
||||
strictly separate authentication paths:
|
||||
|
||||
- Agent routes continue using the existing Bearer token principal.
|
||||
- Human routes use an opaque, expiring HttpOnly cookie backed by a server-side
|
||||
session record and a CSRF token for mutations.
|
||||
|
||||
The interface uses Jinja2 templates, project-owned CSS, and small progressive
|
||||
enhancement JavaScript. No SPA, React, Vue, Node build chain, or duplicated
|
||||
frontend API is introduced.
|
||||
|
||||
## Identity And Registration
|
||||
|
||||
`installation_id` is a UUID generated once by the Agent install/configuration
|
||||
flow and persisted in local Agent state. Registration sends it explicitly and
|
||||
the Hub treats it as the stable installation key. `agent_id` remains the Hub
|
||||
Agent record identity. Hostname, IP, display name, customer code, and company
|
||||
context are metadata and never replace the installation identity.
|
||||
|
||||
Registration and heartbeat accept only allowlisted operational metadata. CNPJ
|
||||
is masked before storage/display, database path is sanitized, and credentials,
|
||||
tokens, passwords, and keys are rejected or discarded.
|
||||
|
||||
## Operational Model
|
||||
|
||||
The Hub stores a current per-installation snapshot rather than recalculating
|
||||
product or event tables on every page request. The snapshot includes product
|
||||
counts, coverage, blocker aggregates, latest job outcomes, heartbeat, health,
|
||||
version drift, company context, Firebird metadata, capabilities, and dataset
|
||||
versions.
|
||||
|
||||
Health is deterministic:
|
||||
|
||||
- `UNKNOWN`: registered but no usable heartbeat/snapshot yet.
|
||||
- `OFFLINE`: last heartbeat is older than the configured threshold.
|
||||
- `FAILED`: persistent technical failure prevents processing.
|
||||
- `DEGRADED`: technical/job failures are relevant but processing remains
|
||||
possible.
|
||||
- `ONLINE`: recent heartbeat with no critical technical failure.
|
||||
|
||||
Fiscal `SAFE_BLOCKED` counts never independently cause `DEGRADED` or `FAILED`.
|
||||
Offline threshold is configurable and defaults to three heartbeat intervals when
|
||||
the configured interval is available.
|
||||
|
||||
## Events, Redaction, And Retention
|
||||
|
||||
Agents submit structured operational events, not raw log files. The Hub
|
||||
redacts case-insensitive secret keys and credentials embedded in strings,
|
||||
limits event message/context size, and persists severity, type, timestamps,
|
||||
installation, optional job/product/blocker references, and structured context.
|
||||
|
||||
Default retention is 90 days via `AGENT_EVENT_RETENTION_DAYS`. A maintenance
|
||||
operation removes only non-audit events older than the retention period. Write
|
||||
audit, blocker history, and critical audit records are preserved.
|
||||
|
||||
## Human Actions
|
||||
|
||||
Human mutations require an authenticated session, valid CSRF token, explicit
|
||||
confirmation in the UI, and an audit event. Allowed operations are reprocess
|
||||
all, reprocess selected products/blockers, resolve a blocker with explicit
|
||||
context, display-name/customer/notes edits, and CSV exports. The panel cannot
|
||||
change Firebird write flags, AI flags, credentials, Firebird paths or CRT,
|
||||
execute SQL, run a shell, or restart a Windows service.
|
||||
|
||||
## APIs And UI
|
||||
|
||||
The panel APIs provide summary, installation list/detail/health/coverage,
|
||||
blockers, jobs, and events with real pagination and filters. Global blockers and
|
||||
jobs reuse the same query services. The UI includes dashboard, agent list,
|
||||
installation detail tabs, global blockers, global jobs, and event timelines.
|
||||
Every primary row uses client, display name, and hostname; UUIDs remain in
|
||||
technical details.
|
||||
|
||||
## Performance
|
||||
|
||||
Indexes cover tenant, installation, health, heartbeat, version, event time and
|
||||
severity/type, job agent/status/time, and blocker agent/status/type/time.
|
||||
Dashboard cards use snapshot aggregates and bounded grouped queries. Detail and
|
||||
global lists use limit/offset pagination for V1, with bounded page sizes and no
|
||||
N+1 relationship loading.
|
||||
|
||||
## Verification
|
||||
|
||||
Tests cover persistent installation identity, sanitized registration and
|
||||
heartbeat metadata, health/offline transitions, snapshot aggregation, event
|
||||
redaction and size limits, retention, admin hashing/login/session/CSRF, API
|
||||
filters and pagination, safe action audit trails, templates, and the existing
|
||||
main/central/agent suites. A non-production fixture generator creates 100
|
||||
varied installations without auto-seeding production.
|
||||
|
||||
## Explicit Non-Goals
|
||||
|
||||
No fiscal rule changes, AI invocation, AI writes, Firebird writes, production
|
||||
deployment, real-client installation, remote Agent update, raw log upload,
|
||||
shell execution, or SPA frontend.
|
||||
+107
@@ -0,0 +1,107 @@
|
||||
# Agent Installation Identity Recovery and First Heartbeat
|
||||
|
||||
## Goal and scope
|
||||
|
||||
Activate the already-enrolled Agent 2 on this Windows host without creating a
|
||||
new Agent, enrolling again, or registering a second installation. Recover and
|
||||
persist its existing Hub `installation_id`, send a first real heartbeat, and
|
||||
stop before any job is claimed. Firebird stays strictly read-only and the
|
||||
installation remains `FIREBIRD_WRITES=0` / `FISCAL_MODE=DRY_RUN`.
|
||||
|
||||
## Current root cause
|
||||
|
||||
`POST /v1/agents/enroll` creates the Hub `Installation` with
|
||||
`installation_id=enrollment_uuid`, but its response omits that identity.
|
||||
`installer/setup_main.py` therefore does not persist it. On a host with no
|
||||
`agent_state.sqlite`, `AgentState.installation_id()` invents a UUID. The Hub
|
||||
heartbeat route rejects a different UUID for an Agent that already has an
|
||||
Installation, preventing the first heartbeat.
|
||||
|
||||
The service loop also calls `GET /v1/jobs/next` immediately after a successful
|
||||
heartbeat, so an explicit runtime gate is required for this activation.
|
||||
|
||||
## Design
|
||||
|
||||
### Identity delivery and recovery
|
||||
|
||||
- Return `installation_id` in the successful enrollment response. The value is
|
||||
the existing `enrollment_uuid` used to create the Hub Installation.
|
||||
- Persist this non-secret value in `agent.env` for new installs.
|
||||
- Add authenticated `GET /v1/agents/me/identity`, returning only
|
||||
`agent_id`, `installation_id`, `hostname`, `version`, and `status`. Keep the
|
||||
existing `/v1/agents/me` response unchanged for compatibility.
|
||||
- At runtime, identity precedence is: existing SQLite `installation_identity`,
|
||||
then configured `INSTALLATION_ID`, then the authenticated identity endpoint.
|
||||
Recovery must verify the returned `agent_id` equals local `AGENT_ID`; absent,
|
||||
malformed, or mismatched identity fails closed without generating a UUID or
|
||||
sending a heartbeat.
|
||||
- Persist a recovered/configured UUID atomically in the existing SQLite state
|
||||
before sending heartbeat. Restarts reuse this stored ID. A conflict between
|
||||
configured and already-persisted values fails closed.
|
||||
|
||||
### Job-poll gate
|
||||
|
||||
- Add `AGENT_JOBS_ENABLED`, defaulting to `1` to preserve current behavior for
|
||||
existing deployments.
|
||||
- When it is `0`, the loop may send heartbeat and sleep, but must not call
|
||||
`next_job`, claim jobs, classify products, or execute jobs.
|
||||
- Set `AGENT_JOBS_ENABLED=0` only for this existing installation while
|
||||
validating the first heartbeat. Leave it disabled after the service is
|
||||
stopped, until a separate operator action authorizes job processing.
|
||||
|
||||
### Deployment boundary
|
||||
|
||||
- The identity endpoint is Hub application code only; it needs no DB migration.
|
||||
Deploy that Hub code without changing the immutable Agent release or the
|
||||
production SSO/DNS configuration.
|
||||
- The installed service binary must contain the identity recovery and job gate.
|
||||
Build only the service-host onedir and update the existing
|
||||
`GTIFiscalAgentService` payload while the service is stopped. Do not run Setup,
|
||||
replace `agent.env`/`credentials.bin`, recreate an Agent, or enroll again.
|
||||
- Keep the published 1.1.1 Setup artifact immutable. The installer source fix
|
||||
gets regression coverage for a future release; this activation updates only
|
||||
the installed service runtime needed by the current host.
|
||||
|
||||
### Write safety
|
||||
|
||||
No Firebird write path is added or enabled. The existing agent configuration
|
||||
must continue to contain `FIREBIRD_WRITES=0`, `FISCAL_MODE=DRY_RUN`, and an empty
|
||||
`AUTHORIZED_WRITE_DATABASE_PATH`. The first heartbeat reports metadata only;
|
||||
this mission must not call `/jobs/next` or a classification endpoint.
|
||||
|
||||
## Failure behavior
|
||||
|
||||
- Identity endpoint unavailable, no existing installation, or Agent ID mismatch:
|
||||
fail closed before heartbeat and before job polling.
|
||||
- SQLite persistence failure: fail closed; do not send a guessed identity.
|
||||
- Jobs disabled: continue the heartbeat loop without touching the job API.
|
||||
- If Hub deployment or the service-host update cannot be completed, do not
|
||||
start the installed service; report the deployment blocker instead.
|
||||
- Service is stopped immediately after the first successful heartbeat is
|
||||
observed and verified against Agent 2's existing installation.
|
||||
|
||||
## Regression coverage
|
||||
|
||||
1. Enrollment response returns the exact `enrollment_uuid` as
|
||||
`installation_id`; Setup persists it without placing any token in `agent.env`.
|
||||
2. Authenticated identity endpoint returns only the approved identity fields
|
||||
and derives hostname/version/status from the existing Agent/Installation.
|
||||
3. Runtime recovers and persists an ID when local state is missing, validates
|
||||
Agent ID equality, reuses the ID on restart, and fails closed on mismatch or
|
||||
missing identity.
|
||||
4. With `AGENT_JOBS_ENABLED=0`, heartbeat runs but `next_job` is never called;
|
||||
default-enabled behavior remains covered.
|
||||
5. Existing write-safety tests continue to prove zero Firebird writes in
|
||||
`DRY_RUN` / `FIREBIRD_WRITES=0`.
|
||||
|
||||
## Acceptance for this activation
|
||||
|
||||
- Existing Agent ID remains 2; no enrollment or Agent creation occurs.
|
||||
- Recovered identity equals the Hub's existing `Installation.installation_id`
|
||||
and is persisted locally before start.
|
||||
- Jobs are disabled before service start; service reaches RUNNING, sends one
|
||||
successful heartbeat for the existing installation, then is stopped.
|
||||
- Panel shows Agent 2 online with the correct hostname and version 1.1.1; no
|
||||
duplicate installation is created.
|
||||
- `FIREBIRD_WRITES=0`, `WRITE_ATTEMPTS=0`, `WRITE_SUCCESSES=0`; no job polling,
|
||||
classification, or Firebird writes occur.
|
||||
@@ -0,0 +1,264 @@
|
||||
# GTI FISCAL WINDOWS INSTALLER EXE V1 — Design
|
||||
|
||||
- Date: 2026-09-23
|
||||
- Status: APPROVED_WITH_ADJUSTMENTS (operator 2026-09-23)
|
||||
- Related: previous mission "Distribuição ZIP V1" shipped release 1.1.0.
|
||||
|
||||
## Objective
|
||||
|
||||
Replace the ZIP download UX with a real Windows `.EXE` installer
|
||||
(`GTIFiscalAgent-Setup-<version>.exe`) plus an installation/enrollment code per
|
||||
installation. One universal installer per version. Publish release 1.1.1 in the
|
||||
production Hub, run a REAL Windows E2E, and stop before the first client.
|
||||
|
||||
Acceptance (operator visible): Fiscal → Agents → Nova instalação → Criar →
|
||||
Baixar Agent → browser downloads `GTIFiscalAgent-Setup-1.1.1.exe` directly (an
|
||||
executable, not a ZIP). Run the EXE on Windows, type the code, install/enroll.
|
||||
ZIP as the main flow = FAIL.
|
||||
|
||||
## Approved adjustments (operator)
|
||||
|
||||
1. Setup.exe uses `requireAdministrator`/`--uac-admin` manifest. NO hybrid
|
||||
asInvoker + self-elevation flow.
|
||||
2. Setup.exe is self-contained: embeds the Service host + runtime + Uninstall.exe.
|
||||
It never downloads a second ZIP during install.
|
||||
3. `install_code` alone resolves the Installation on the Hub. User types only the
|
||||
code. If both `install_code` and `enrollment_token` are present, the request
|
||||
is rejected.
|
||||
4. Production: the permanent credential NEVER goes into `agent.env`. It lives
|
||||
only in `credentials.bin`, protected by DPAPI LocalMachine + ACL
|
||||
SYSTEM/Administrators. `agent.env` holds only non-secret configuration plus
|
||||
`FIREBIRD_WRITES=0`.
|
||||
5. "Reproducible build" = an automated, repeatable build process and an
|
||||
immutable release — NOT byte-identical hashes across rebuilds. After
|
||||
PUBLISHED, 1.1.1 can never be replaced/overwritten.
|
||||
|
||||
Also fixed in this design:
|
||||
- 16 Crockford Base32 chars = 80 bits of entropy for the install code.
|
||||
- Enrollment rate limit keyed by IP AND by code/installation.
|
||||
- Install code is normalized (hyphens/case) before hashing.
|
||||
- Release 1.1.0 stays untouched.
|
||||
- `FIRST_REAL_CLIENT_AGENT_INSTALLED=false` this mission.
|
||||
|
||||
## Installer technology
|
||||
|
||||
**Chosen: PyInstaller (all-Python).** No Inno Setup.
|
||||
|
||||
Reasoning (INSTALLER_TECHNOLOGY / INSTALLER_REASON):
|
||||
- PyInstaller 6.21 is already installed on the Windows dev machine; Inno Setup is
|
||||
not. A fully automated, fail-fast build is achievable today with `pip`-managed
|
||||
tools.
|
||||
- The enrollment wizard already needs HTTPS (httpx) and DPAPI (pywin32
|
||||
win32crypt); those are Python layers, so the wizard must be Python regardless.
|
||||
Adding Inno Setup would add a second tool without removing the Python wizard
|
||||
requirement.
|
||||
- pywin32 gives the battle-tested Windows Service wrapper, and PyInstaller's own
|
||||
runtime sulution replaces a venv in Program Files, so no Python/pip is needed
|
||||
on clients.
|
||||
- The previous mission already validated the service wrapper + `sc.exe failure`
|
||||
recovery path in the pilot checklist.
|
||||
- Reproducible build is achieved with a dedicated PowerShell build script (clean
|
||||
output dir, official VERSION, SHA256, metadata, fail on missing dep); exact
|
||||
byte-identical output is NOT required.
|
||||
|
||||
### Artifacts (universal; built on the Windows dev machine)
|
||||
|
||||
| Artifact | PyInstaller mode | Contents / role |
|
||||
|---|---|---|
|
||||
| `GTIFiscalAgent-Setup-<v>.exe` | onefile, `--uac-admin` | GUI wizard (tkinter). Embeds Agent runtime sources + service host + `Uninstall.exe` as bundled data. Enroll → write config → DPAPI credential → extract Service host → register service → preflight. Main download artifact. |
|
||||
| `GTIFiscalAgentService.exe` (+ `_internal/`) | onedir | The Windows Service host (bundled inside Setup.exe, extracted to `C:\Program Files\GTI\FiscalAgent\`). Reuses `gti_agent_windows_service.py` logic; token comes from `credentials.bin`. |
|
||||
| `Uninstall.exe` | onefile | Stops/removes the service, removes Program Files tree + ARP entry. Invoked from Settings → Apps → GT1 Fiscal Agent → Uninstall. |
|
||||
|
||||
Setup.exe behavior:
|
||||
1. Program runs elevated (requireAdministrator manifest). If not elevated,
|
||||
Windows UAC prompts.
|
||||
2. tkinter window: title "GTI Fiscal Agent", a single install-code field, Hub
|
||||
URL fixed at `https://gtifiscal.gtigrupo.com.br` (no prompt; dev override via
|
||||
env / CLI flag only).
|
||||
3. "Instalar" → POST `/v1/agents/enroll` `{"install_code": "XXXX-XXXX-XXXX-XXXX"}`.
|
||||
4. On success (agent_id, agent_token, hub_url, tenant, release info):
|
||||
- Write `agent.env` (non-secret): GTI_HUB_URL, AGENT_ID, COMPANY_ID/UF
|
||||
(optional), FIREBIRD_DATABASE_PATH (blank), FB_USER/FB_PASSWORD blank,
|
||||
FISCAL_MODE=DRY_RUN, FIREBIRD_WRITES=0, AGENT_VERSION, POLL_INTERVAL_SECONDS,
|
||||
AGENT_LOG_PATH, AGENT_STATE_DB, EXPECTED_SCHEMA_FINGERPRINT.
|
||||
- Write `credentials.bin` via DPAPI LocalMachine encryption
|
||||
(`win32crypt.CryptProtectData(..., CRYPTPROTECT_LOCAL_MACHINE)`); contains
|
||||
AGENT_TOKEN (and optional secrets). Apply ACL: SYSTEM/Administrators only
|
||||
(icacls). `agent.env` never holds the token.
|
||||
- Extract bundled Service host onedir to `C:\Program Files\GTI\FiscalAgent\`.
|
||||
- Install service `GTIFiscalAgent` (Automatic/Delayed), set recovery
|
||||
`sc.exe failure "GTIFiscalAgent" reset= 86400 actions= restart/60000/restart/60000/restart/60000`.
|
||||
5. Preflight: Firebird path not configured → report "serviço instalado; configure
|
||||
o arquivo de banco para ativar" and do NOT start the service (no error loop).
|
||||
If fully configured → optionally start.
|
||||
6. Write ARP uninstall key (DisplayName "GTI Fiscal Agent",
|
||||
UninstallString = Uninstall.exe).
|
||||
7. `WINDOWS_INSTALL_COMPLETED` audit (best effort) once install succeeds.
|
||||
|
||||
The installer never stores the enrollment code after success, never embeds any
|
||||
permanent credential, and never enables Firebird/AI writes (no UI to do so).
|
||||
|
||||
## Installation code (`install_code`)
|
||||
|
||||
- Format: 16 chars, Crockford Base32 alphabet
|
||||
(`0123456789ABCDEFGHJKMNPQRSTVWXYZ` — excludes I, L, O, U), grouped
|
||||
`XXXX-XXXX-XXXX-XXXX`. 16 chars × 5 bits = 80 bits entropy.
|
||||
- Generation: `secrets.randbelow`/random pick of 16 chars; return formatted.
|
||||
- Normalization before hashing: uppercase, strip hyphens/spaces.
|
||||
- Storage: only SHA-256 of the normalized code. Never the plaintext.
|
||||
- Columns on `agent_enrollments` (migration `0023`):
|
||||
`install_code_hash` (str 64), `install_code_expires_at` (timestamptz),
|
||||
`install_code_attempts` (int, default 0), `install_code_created_at` (timestamptz).
|
||||
- TTL: `AGENT_ENROLLMENT_TTL_SECONDS` (default 86400) reused.
|
||||
- One active code per installation at a time; "Gerar novo código" rotates it
|
||||
(old code invalidated, previous hash kept for audit but unusable).
|
||||
- Revocation (`Revogar`) invalidates the code and, when an agent exists,
|
||||
deactivates it.
|
||||
- Rate limit: `rate_limit.record_attempt(bucket="enroll_code:"+normalized_code_hash or agent_enrollment_id, origin=IP)` in addition to per-IP bucket.
|
||||
|
||||
## Enrollment endpoint (`POST /v1/agents/enroll`)
|
||||
|
||||
- Body: `{"install_code": "..."}` OR legacy `{"installation_id": ..., "enrollment_token": ...}`.
|
||||
- If both `install_code` and `enrollment_token` are provided → 400, `ENROLLMENT_REJECTED` (reason INVALID_REQUEST).
|
||||
- With `install_code` (primary):
|
||||
- Resolve the Installation by normalized-code hash. No installation_id needed.
|
||||
- Validate: exists, status=PENDING (not REVOKED), release PUBLISHED, code not
|
||||
expired, one-time. `install_code_attempts` guard + rate limit.
|
||||
- On success: digest the code (mark used), set installation status ENROLLED,
|
||||
create the Agent with a fresh permanent token, return
|
||||
`{agent_id, agent_token, installation_id, hub_url, ...}`.
|
||||
- `ENROLLMENT_STARTED / SUCCEEDED / REJECTED` audit; never log the code.
|
||||
- Legacy `enrollment_token` path kept for backward compatibility (published 1.1.0
|
||||
bundles) but is no longer produced for new installations.
|
||||
|
||||
## Downloads
|
||||
|
||||
- `/agents/releases` main button + pending detail + temp link: serve the SAME
|
||||
universal EXE (`GTIFiscalAgent-Setup-<v>.exe`) stored as the release artifact.
|
||||
- Filename/querystring carry no code/token.
|
||||
- `INSTALLER_DOWNLOADED` audit event on download.
|
||||
|
||||
## Release model
|
||||
|
||||
- New release row: `version=1.1.1`, `platform=windows`, `architecture=x64`,
|
||||
`filename=GTIFiscalAgent-Setup-1.1.1.exe`, `status=PUBLISHED`,
|
||||
`published_at` (per the prior fix), `sha256`, `size_bytes`.
|
||||
- Storage path generalized (EXE artifact, not hardcoded `.zip`). The internal
|
||||
ZIP remains a build/debug artifact only (kept under dist/, not the main flow).
|
||||
- Release 1.1.0 row and artifact remain untouched.
|
||||
- After PUBLISHED, the artifact is immutable: publish script refuses to replace
|
||||
an existing PUBLISHED filename for the same version+platform+arch.
|
||||
|
||||
## Agent runtime changes (FIREBIRD_WRITES=0, DPAPI credential)
|
||||
|
||||
- New module `gti_fiscal_agent/credential_store.py`:
|
||||
- `read_credential(path)` — decrypt DPAPI LocalMachine blob (fallback: if the
|
||||
file is a plaintext token, accept in dev only).
|
||||
- `write_credential(path, payload)` (installer side; reused by Setup.exe) —
|
||||
encrypt + set ACL.
|
||||
- `install_env_from_credentials(path, env)` — fill `AGENT_TOKEN` from
|
||||
`credentials.bin`; `agent.env` stays non-secret.
|
||||
- `Config`/`main`/service wrapper read the token via
|
||||
`AGENT_TOKEN` if set, else look for `GTI_CREDENTIAL_FILE` (default
|
||||
`C:\Program Files\GTI\FiscalAgent\credentials.bin`) and decrypt.
|
||||
- `FIREBIRD_WRITES=0` enforced (already fail-closed in `Config.firebird_writes`
|
||||
and `open_write_connection`); drift test asserts the installer template sets 0
|
||||
and no UI/flag enables it.
|
||||
|
||||
## Panel UI changes
|
||||
|
||||
- `agent_pending_detail.html` + `agent_detail.html`: show
|
||||
- Versão, Status,
|
||||
- Código de instalação `XXXX-XXXX-XXXX-XXXX` + [Copiar]
|
||||
- Expira em (formatted TTL deadline)
|
||||
- [Baixar instalador (.EXE)] → EXE
|
||||
- [Gerar novo código] → rotates code (audit INSTALLATION_CODE_CREATED)
|
||||
- [Revogar] → revokes (audit INSTALLATION_CODE_REVOKED / INSTALLATION_REVOKED)
|
||||
- `/agents/releases`: download button → EXE (main flow).
|
||||
- Never show raw enrollment tokens; the install code is shown to the operator who
|
||||
created the installation only (SSO-authenticated panel).
|
||||
|
||||
## Build script
|
||||
|
||||
- `scripts/build_windows_installer.ps1`
|
||||
- Reads official `release/VERSION` (1.1.1).
|
||||
- Cleans `dist/installer` output.
|
||||
- Builds service host (onedir) and Setup.exe (onefile, `--uac-admin`) and
|
||||
Uninstall.exe with PyInstaller from the bundled sources; Setup embeds the
|
||||
service host dir + Uninstall.exe as data.
|
||||
- Computes sha256, writes `GTIFiscalAgent-Setup-<v>.exe.sha256`.
|
||||
- Fails on missing dependency/tool.
|
||||
- Deterministic inputs; byte-identical output not required.
|
||||
|
||||
## Tests (TDD)
|
||||
|
||||
Central (`central/tests`, pytest):
|
||||
- install code generation/normalization/hash round trip (format, alphabet,
|
||||
uppercase, hyphen removal, no ambiguous chars).
|
||||
- enroll with `install_code` resolves installation by code alone; success creates
|
||||
enrolled agent + credential issuance; `install_code_attempts` increments;
|
||||
expiry/revoked/used-one-time → REJECTED; code+token together → 400;
|
||||
rate limit by IP and by code; duplicate/rotation invalidation.
|
||||
- migration 0023 up/down.
|
||||
- publish script refuses to overwrite a PUBLISHED artifact and stamps
|
||||
`published_at`.
|
||||
- downloads serve EXE with `INSTALLER_DOWNLOADED` audit.
|
||||
- panel pages render code/button actions without leaking tokens.
|
||||
|
||||
Agent (`agent/tests` + root `tests`): credential_store encrypt/decrypt, DRY_RUN /
|
||||
FIREBIRD_WRITES=0 enforcement, `agent.env` has no token, service wrapper reads
|
||||
credential.
|
||||
|
||||
Build verification: PowerShell script dry-run on dev machine produces EXE +
|
||||
sha256; PyInstaller starts the setup binary (elevation triggered) or we verify
|
||||
artifact layout.
|
||||
|
||||
## E2E plan
|
||||
|
||||
1. Local: build 1.1.1 EXE; launch against the PRODUCTION Hub once the backend is
|
||||
deployed (or against a local Hub in a scratch container for the UI pass).
|
||||
2. Production Hub (real, at `https://gtifiscal.gtigrupo.com.br`):
|
||||
- Create an E2E installation via the panel.
|
||||
- Generate an install code.
|
||||
- Download the real EXE (assert filename, MIME application/octet-stream /
|
||||
executable, not ZIP).
|
||||
- Run on the real Windows dev machine, enter code, complete enrollment
|
||||
(real `POST /v1/agents/enroll`), verify:
|
||||
- Installation → ENROLLED, Agent created.
|
||||
- `credentials.bin` exists (DPAPI-encrypted, not plaintext token).
|
||||
- `agent.env` has no token, FIREBIRD_WRITES=0, blank Firebird path.
|
||||
- Service `GTIFiscalAgent` registered, Automatic/Delayed, recovery set,
|
||||
NOT started (Firebird not configured) — or started if configured.
|
||||
- Windows firewall/dir ACL ok (SYSTEM/Administrators).
|
||||
- Uninstall via Uninstall.exe → service removed, Program Files cleaned.
|
||||
- Revoke/deactivate the E2E agent in the panel; keep audit trail.
|
||||
- No real Firebird database is ever connected.
|
||||
3. Rejections E2E: invalid code, expired code, replayed/used code (each audited
|
||||
ENROLLMENT_REJECTED), code+token together → 400.
|
||||
|
||||
## Deploy sequence
|
||||
|
||||
1. Backup production: PostgreSQL logical dump (`pg_dump`) of `gti_fiscal` +
|
||||
copy of release storage volume (via container) to the backup location.
|
||||
2. Build image with migration `0023`; `alembic upgrade head` (fresh-DB and
|
||||
head checks).
|
||||
3. Deploy Hub (overlay, docker compose up -d --build), postgres intact.
|
||||
4. Publish release 1.1.1 EXE (immutable).
|
||||
5. Production smoke: health endpoint, panel pages, download EXE.
|
||||
6. Run production E2E (above).
|
||||
7. Validate restore: restore EXE artifact + PG metadata from backup copy.
|
||||
|
||||
## Constraints (non-negotiables)
|
||||
|
||||
- FIREBIRD_WRITES=0; AI_WRITE_ALLOWED=false; no UI enabling either.
|
||||
- No real client install this mission.
|
||||
- No SSO/NPM/DNS/hardware/credentials changes.
|
||||
- Release 1.1.0 not touched. 1.1.1 immutable once PUBLISHED.
|
||||
- CODE_SIGNED=false. SmartScreen warning acceptable in the internal pilot.
|
||||
- Credentials never logged/committed; `GTI_SSH_PASSWORD` env var only.
|
||||
|
||||
## Report (final)
|
||||
|
||||
Populate the mission's exact field list (STATUS, INSTALLER_TECHNOLOGY=PyInstaller,
|
||||
INSTALLER_REASON, ..., PANEL_DOWNLOAD_FILENAME=GTIFiscalAgent-Setup-1.1.1.exe,
|
||||
DPAPI=TRUE, ACL=system+administrators, FIREBIRD_WRITES=0, WINDOWS_E2E=..., etc.).
|
||||
@@ -0,0 +1,41 @@
|
||||
# GTI Fiscal — painel operacional com drill-down
|
||||
|
||||
**Objetivo.** Refinar as telas atuais Resumo, Agents, Blockers, Jobs e Nova instalação para operação de ~70 clientes, inclusive clientes com 20.000+ produtos. Preservar visual do Gerenciador, stack Jinja/FastAPI/SQLAlchemy e funcionalidades fiscais, Agent, SSO e sincronização existentes. Sem gráficos decorativos, framework frontend novo nem carga de produtos no browser.
|
||||
|
||||
## Semântica operacional e histórico
|
||||
|
||||
Uma expressão SQL compartilhada define `OPERATIONALLY_ACTIVE` para instalação vinculada a Agent: `Tenant.active IS TRUE`, `Agent.enabled IS TRUE` e ausência de enrollment vinculado em estado `REVOKED`, `EXPIRED` ou `FAILED` (estados finais/inativos). Agent legado sem enrollment não é revogado por inferência; clientes `LOCAL` não são classificados por nome. `Installation.processing_enabled=false` pausa processamento, **não** equivale a instalação revogada: ainda pode comunicar heartbeat e indicar indisponibilidade. Nunca excluir registros por nome `E2E` ou por versão antiga. O registro histórico permanece intacto.
|
||||
|
||||
Dashboard usa exclusivamente esta expressão para contagem de saúde, soma de snapshots/cobertura, blockers operacionais, jobs falhos operacionais, versão desatualizada e atenção. Blockers ativos requerem instalação vinculada operacionalmente ativa e `resolved_at IS NULL`; blockers legados sem instalação e blockers resolvidos permanecem nas listagens históricas. A página Agents usa `scope=active` por padrão e `scope=all` para inventário/histórico (com indicação de histórico e badges); Blockers/Jobs usam o mesmo `scope` para a lista e seus KPIs, preservando filtros adicionais. Um card do Resumo sempre leva à visão `scope=active` e ao filtro correspondente. O filtro `scope=all` inclui registros inativos/revogados, e `status=RESOLVED` em Blockers permite acessar problemas resolvidos. Totais do dashboard e contagens da listagem ativa são reconciliados pela mesma expressão SQL.
|
||||
|
||||
## Resumo: decisão em um clique
|
||||
|
||||
Seis cards `<a>` completos para Total, Online, Degraded, Offline, Failed, Unknown; destino `/agents?scope=active` com `health=<ENUM>` quando aplicável. Labels de apresentação PT-BR: Servidores, Online, Atenção, Offline, Falhou, Não informado. A identidade técnica do filtro permanece na URL e na query SQL. Cobertura: total/completos levam a Agents com ordenação fiscal (visão disponível), bloqueados levam a `/blockers?scope=active&status=OPEN`, falhas técnicas a `/agents?scope=active&health=FAILED`, e não classificados a Agents ordenados por pior cobertura. Sem prometer filtros de produto que o sistema não possui. O breakdown CFOP/ST/ICMS/CSOSN/NCM/OTHER liga valores positivos a `/blockers?scope=active&status=OPEN&field=<ENUM>`; zeros ficam visualmente neutros.
|
||||
|
||||
“Atenção necessária” usa até 10 **grupos**, não 10 linhas cruas: primeiro instalações com falha técnica, depois saúde Failed/Degraded/Offline, grupos de blockers fiscais por instalação/campo/tipo/motivo, versões desatualizadas e outros estados realmente acionáveis. Dentro do grupo fiscal: número de blockers abertos e produtos distintos; agrupar **em SQL antes do LIMIT**, com ordenação por gravidade e recência. Um grupo oferece link de lista filtrada de blockers ou detalhe de Agent/job. Instalações revogadas, agentes desabilitados, clientes inativos, itens resolvidos ou sem destino operacional não aparecem. Link “Ver todos” abre lista operacional correspondente. Evitar duplicar a mesma instalação como alerta de versão quando um alerta técnico de prioridade maior já a representa; histórico continua nas listas.
|
||||
|
||||
Jobs falhos: contagem e lista curta da operação ativa com Cliente, Agent/hostname, tipo, erro truncado, momento e link `Ver job` para `/jobs?scope=active&status=FAILED&job_id=<id>`. A página Jobs exibe erro integral quando `job_id` é informado. Link “Ver jobs” abre `status=FAILED`. Agents desatualizados: lista compacta em largura suficiente (sem sidebar estreita), cliente/hostname, versão atual/atualização e detalhe do Agent; poucos itens mais “Ver todos” com `version=outdated` aplicado.
|
||||
|
||||
## Agents / Servidores
|
||||
|
||||
Uma tabela densa, legível em 1366×768 sem scroll horizontal: cliente com instalação/hostname na mesma célula, saúde, fiscal (quantidades resumidas), cobertura, produtos, versão e último contato. Texto secundário só quando existir; badges mostram apenas o valor humano, sem repetir “Saúde técnica:” ou “Status fiscal:”. Busca, saúde (todos os cinco estados), ordenação e scope via GET; chip de filtro ativo e “Limpar filtros”; paginação mantém todos os parâmetros. Link primário visível no cliente/hostname é nativamente acessível por Tab/Enter; clique com mouse na restante da linha abre o mesmo detalhe via delegação pequena em JS, sem eliminar a semântica da tabela. Em telas pequenas, linhas podem assumir apresentação compacta com rótulos por célula, mantendo filtros e ações expostos.
|
||||
|
||||
## Blockers
|
||||
|
||||
KPIs coerentes com `scope`, `status` e demais filtros (abertos, produtos/clientes afetados, resultado filtrado), com links apenas quando a métrica tem destino fiel. Breakdown filtrado liga campo positivo ao mesmo conjunto de filtros, trocando só `field`; filtro selecionado fica visível. Filtros existentes permanecem server-side (cliente, instalação, campo, tipo, status, idade, busca e identificadores existentes), dispostos em uma ou duas linhas no desktop e empilhados no mobile, com “Limpar filtros” retornando à visão ativa padrão.
|
||||
|
||||
Tabela destaca código e descrição do produto como link para `/blockers/<id>`; exibe cliente/servidor, campo, tipo humano, prioridade humana, problema humano, último registro e CTA. Tradutor somente de apresentação mapeia enums conhecidos (`BLOCKED_MISSING_EVIDENCE` → `Evidência fiscal ausente`, `INVALID_NCM` → `NCM inválido`, prioridades `CRITICAL/HIGH/MEDIUM/LOW` → `Crítica/Alta/Média/Baixa`) e cai em rótulo legível para desconhecidos; o detalhe mantém o código técnico original. Paginação limitada (50), consulta com COUNT separado, sem dropdown de produtos nem N+1.
|
||||
|
||||
## Jobs
|
||||
|
||||
Atalhos compactos para `PENDING`, `RUNNING`, `COMPLETED`, `FAILED`, todos filtrando `scope=active` e mostrando a seleção. Status PT-BR: Pendente, Executando, Concluído, Falhou; tipos conhecidos (`SYNC_INCREMENTAL`, `INITIAL_SYNC`, `REPROCESS_ALL`, `REPROCESS_PRODUCTS`) ganham labels humanos. A lista agrupa cliente e servidor, depois tipo, status, criado/finalizado, duração e erro curto. Erro integral e enum técnico aparecem na vista filtrada `job_id` da própria página, sem criar uma tela artificial. Seleção de scope histórico e “Limpar filtros” mantêm os demais parâmetros úteis. Não acionar jobs, somente leitura do banco.
|
||||
|
||||
## Estilo, acessibilidade e escala
|
||||
|
||||
Usar somente `central/static/control-panel.css` sobre os tokens imutáveis `gti-theme.css`. Estados semânticos: sucesso verde, atenção amarelo, falha vermelha, histórico/desconhecido cinza, bloqueio fiscal laranja. Foco visível, navegação Tab/Enter e Space nos controles apropriados, links `<a>` em cards; nenhum `div` simulando botão. Mobile 375 px preserva busca e ações, desktop 1366×768 e 1920×1080 aproveitam largura sem blocos de vazio. Testar light/dark por screenshot, não só inspecionando tokens. Combobox de clientes ativos da Nova instalação permanece com busca nome/CNPJ, mouse/teclado, `client_id` hidden e validação server-side; só retoque estético se necessário.
|
||||
|
||||
Queries devem usar agregados e grupos SQL, bounded LIMIT, COUNT e paginação server-side; não carregar listas de produtos/snapshots inteiros nem emitir uma consulta por item. Medir número de queries do dashboard e inspecionar planos quando filtros/campos precisarem de índice. Se necessário, migration de índices pequena e reversível; nunca mudar engine fiscal ou dados históricos. Regressões confirmam exclusão de inativos/revogados, coerência card→querystring→resultado, agrupamento, labels, paginação e query count/ausência de N+1.
|
||||
|
||||
## Validação e deploy
|
||||
|
||||
Rodar suites existentes e regressões focadas. Browser smoke em Resumo, Agents, Blockers, Jobs, Nova instalação; navegar Resumo→Online/Total/NCM/Jobs falhos/Agent desatualizado e voltar; verificar light/dark nos viewports 1920×1080, 1366×768, 375 px e Tab/Enter/Space. Nova instalação não deve criar registros no smoke. Usar **exclusivamente** `scripts/deploy_hub.ps1` para deploy/backup/migration/rollback/health, sem SSH improvisado nem scripts paralelos. Verificar health público e métricas operacionais pós-deploy. Se autenticação visual do Gerenciador não estiver disponível, reportar explicitamente o bloqueio em vez de alegar screenshot de produção.
|
||||
@@ -0,0 +1,67 @@
|
||||
# cClassTrib applicability gate
|
||||
|
||||
## Purpose
|
||||
|
||||
Stop treating an empty `C000025.CCLASSTRIB` as a fiscal defect solely because
|
||||
the global RTC feature is enabled. Keep the single existing fiscal-completeness
|
||||
result and continue evaluating every other fiscal field normally.
|
||||
|
||||
## Evidence and selected applicability rule
|
||||
|
||||
The SGV database has `PARAMETROS.USAR_REFORMA_TRIBUTARIA=1` and RTC-capable
|
||||
schema, but the inspected `PARAMETROS` and `C000004` schemas have no separate
|
||||
cClassTrib enable flag. The SGV product screen exposes `C000025.CCLASSTRIB` as
|
||||
an editable catalog-backed field; available evidence does not mark it required
|
||||
for every product. The current code incorrectly promotes global RTC capability
|
||||
to universal per-product requiredness.
|
||||
|
||||
Use the live company regime (`C000004.CRT`) together with the documented
|
||||
effective-date gate:
|
||||
|
||||
- RTC capability/`USAR_REFORMA_TRIBUTARIA` must still be enabled for RTC
|
||||
evaluation generally.
|
||||
- CRT 3: cClassTrib is applicable from 2026-08-03.
|
||||
- CRT 1, 2, or 4: cClassTrib is applicable from 2027-01-01.
|
||||
- Missing, conflicting, or unsupported CRT: cClassTrib is not applicable for
|
||||
completeness until the company context is resolved; never infer a regime.
|
||||
- Before the applicable date, ignore cClassTrib completely: no blocker, no
|
||||
candidate, no completeness penalty, and no write.
|
||||
- Once applicable, preserve the existing validation/resolution behavior,
|
||||
including a blocker for an empty value when no safe resolution is available.
|
||||
|
||||
This makes current Agent 2 (`CRT=1`, `OPTANTE_SIMPLES=S`) not applicable on
|
||||
2026-09-25 even though `USAR_REFORMA_TRIBUTARIA=1`. Its current RTC write mode
|
||||
(`DRY_RUN`) is not changed.
|
||||
|
||||
## Minimal implementation boundary
|
||||
|
||||
Add the gate to the existing shared fiscal evaluator/context using the already
|
||||
resolved live company CRT and current date. Do not add a new SGV flag, table,
|
||||
panel, or second coverage metric. Keep PIS/COFINS, NCM, resolver rules, RTC
|
||||
write authorization, and Agent write allowlist unchanged. Because the evaluator
|
||||
is executed by the Agent, this is an Agent code change. After test completion,
|
||||
the operator explicitly authorized release 1.2.5 through the official Hub/Agent
|
||||
deploy path and an in-place Agent 2 upgrade that preserves its identity.
|
||||
|
||||
The RTC page should present the existing single-panel status as
|
||||
“cClassTrib: Não utilizado neste cliente” while the gate is false.
|
||||
|
||||
## Verification
|
||||
|
||||
Add the requested focused tests: not-applicable with empty cClassTrib (RTC off
|
||||
and on), unrelated fiscal fields still evaluated, applicable with empty
|
||||
cClassTrib still blocked, and zero write attempts. Then run a read-only
|
||||
`REPROCESS_ALL` against Agent 2 and compare its latest result/snapshot and
|
||||
blockers with the captured baseline. Preserve `RTC_WRITE_MODE=DRY_RUN` and
|
||||
verify Firebird write attempts/successes remain zero. Report NCM/PIS/COFINS
|
||||
blockers independently.
|
||||
|
||||
## Baseline captured before implementation
|
||||
|
||||
Agent 2 latest Hub snapshot/job: total 521, complete 0, safe-blocked 521,
|
||||
failed 0, unclassified 0, coverage 0%; RTC blockers 521; PIS blocker products
|
||||
1; COFINS blocker products 1; NCM blocker products 26 (29 open blocker rows).
|
||||
The offline read-only Firebird discovery saw `USAR_REFORMA_TRIBUTARIA=1`,
|
||||
`SISTEMA_TODO_FISCAL=0`, `C000004.CRT=1`, and
|
||||
`C000004.OPTANTE_SIMPLES='S'`; its guarded query recorded zero write attempts
|
||||
and zero successes.
|
||||
Reference in New Issue
Block a user