Documents what the code already does (no behavior change): GTI_FISCAL_VERSION
(Hub) and the Agent's own version are separate numbers with separate release
cycles; central.app.installation_identity.reference_agent_version() already
compares an Agent's heartbeat version against the highest PUBLISHED Windows
AgentRelease, never against the Hub's version - verified against
test_version_reference.py. Also documents that there is no real minimum-
version connection gate today (only SCHEMA_FINGERPRINT is enforced) and where
one would go if ever needed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missão 4 / Bloco B.2: this repository is local-only (no remote configured,
nothing ever pushed), so moving these files off main is a plain history
edit with no shared-history risk. agent/src/gti_fiscal_agent/cfop_ippt_auto_apply.py,
its tests and docs/CFOP_IPPT_WRITE_CAPABILITY_V1.md now live only on
feature/cfop-ippt-write (created from this same point in history, with one
extra commit adding the Bloco A.3 end-to-end regression test). Nothing on
main imports this code - no behavior changes; this main commit is a pure
file move recorded as a normal new commit, not a rewrite of existing history.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missão 4 / Bloco A.1 (prioridade máxima): central.app.field_contract.infer_role
treated ANY C000032.TIPO=COMPOSICAO row for a product as proof of own
production, with no ratio check - a beer sold 19 times and used once as a
recipe component (Agent 2, real data) was classified OWN_PRODUCTION from
that single occurrence, proposing to change its CFOP and IPPT.
New deterministic threshold in field_contract.py: COMPOSICAO must be a
majority (>=70%) of that product's own C000032 movement history, with at
least 5 total events, to count as own-production evidence. Below 30% the
evidence is discarded entirely (falls through to other role evidence,
e.g. TIPO=VENDA -> RESALE). Between 30% and 70% (with enough events to be
meaningful) is genuinely conflicting evidence -> UNKNOWN, never guessed
either way. RECEITA_INGREDIENTES/PROD_COZINHA-based evidence (a direct
recipe-table linkage, not a statistical ratio) is untouched.
agent/src/gti_fiscal_agent/firebird_reader.py now counts COMPOSICAO vs other
C000032 rows per product (read_product_evidence: single indexed query;
read_evidence: one grouped query for the whole catalog) into a new
ev["composed_ratio"] key, alongside the existing "composed" set (kept for
other, non-role consumers like coverage.py). central/app/enrichment/api.py's
sanitize_product_evidence and fiscal_completion.py's empty-ev fallback both
carry the new key through.
New tests (central/tests/test_field_contract.py) cover the 5 real cases named
in the mission: the beer (1 vs 19, 5% - discarded), a second beer (3 vs 4,
42.9% - MIXED/UNKNOWN), the liqueur (34 vs 22, 60.7% - still MIXED/UNKNOWN,
below the 70% bar), the juice with PROD_COZINHA=1+RECEITA_INGREDIENTES
(unaffected, stays OWN_PRODUCTION), and the tilapia/corvina fish (no
composed evidence at all, stays RESALE) - plus a strong 90% majority case
and an insufficient-sample (<5 events) case. A new end-to-end test in
agent/tests/test_cfop_ippt_auto_apply.py (committed separately, see the
feature/cfop-ippt-write branch) proves the weak-evidence beer never reaches
select_candidates as a write candidate, independent of any write flag.
Re-running the dry-run over Agent 2's real database (GTI.FDB, 527 products)
after this fix: AUTO_FILL_SAFE CFOP/IPPT cells drop from 12 to 3 (the two
fish CFOP fills + the juice's IPPT, all with strong, non-statistical
evidence) - docs/runs/GTI_dryrun_universe_summary_after_A1.json. This
corrects the Missão 3 report's 12/9(downgraded)/4(sustained) manual
judgment call with a deterministic, tested engine rule.
No schema/migration change. Zero Firebird writes (read-only analysis only).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Hub-only version bump for the static dossie_contador.md anonymization fix
and the Bloco B PIS/COFINS-source evidence update committed just before.
No schema/migration change, no Agent file touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missao 3 / Bloco C.1-C.2: new agent/src/gti_fiscal_agent/cfop_ippt_auto_apply.py
mirrors the existing rtc_auto_apply.py safety model for a new, narrower
capability (CFOP_IPPT_WRITE_V1): compare-before-write, stale-write guard
("human override sempre vence" - a value changed since the candidate was
selected is never overwritten), one transaction per cell, readback
verification, and rollback that only restores a value still equal to what
was written. Candidates come exclusively from field_contract.resolve_product's
own AUTO_FILL_SAFE classification for CFOP/IPPT - no new fiscal rule.
14 new tests (agent/tests/test_cfop_ippt_auto_apply.py) prove: flag OFF means
the caller never reaches a write; only CHANGE+write_eligible CFOP/IPPT cells
become candidates; a human edit between decision and write blocks the write;
a second run over an already-applied cell writes zero; a technical failure on
one cell never stops the batch; rollback refuses when the value changed again.
Deliberately NOT wired into agent/scripts/run_agent_service.py::execute_job,
and the Hub-side authorization/audit plumbing (Installation.cfop_ippt_write_mode,
audit table, job-delivered authorization - the same pattern as
central/app/rtc_audit.py) does not exist yet. No currently-published Agent
behavior (1.3.8) changes with this commit - nothing imports this new module.
See docs/CFOP_IPPT_WRITE_CAPABILITY_V1.md for the acceptance roteiro (test
products only, never real ones) and what remains before this can ever be
activated for a real installation - deliberately left for review before any
further step, per this mission's explicit instruction not to publish it active.
Also: dossie_contador.md (internal + anonymized) updated with the Bloco B
finding that the ~9.5% CST=01 anomaly is confined to a single calendar month
(not the whole year), with no fiscal-dimension (NCM/CFOP/CSOSN/ICMS group)
explaining which items got it - consistent with a one-off technical event,
not a deliberate rule.
REAL_PRODUCT_FISCAL_WRITES=0 (no write path exists in this commit).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New script (scripts/blocoB_pis_cofins_source.py) and evidence artifact
(docs/runs/BLOCOB_PIS_COFINS_SOURCE_V1.json): tests whether NCM, CODGRUPO, CFOP,
CSOSN or GRUPO_ICMS determine the CST_PIS_COFINS value emitted in NFCE_ITENS,
across all 5 installations. Read-only; no Firebird write path exists in this script.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missao 2 (fechamento de lacunas fiscais, read-only + UI/engine gated):
- field_contract: pis_cofins_profile now also accepts the literal sentinels
NOT_APPLICABLE/OPERATION_TIME, resolving to NA only from an audited
FiscalContextEntry - never inferred from emission statistics. CFOP_ENTRADA
under an already-decided entry_scope=MIXED_OR_DYNAMIC now resolves to NA
instead of MANUAL, so it stops blocking product completeness and stops
re-asking an already-answered question. Regression tests prove: no ADMIN
decision => unchanged old behavior; decision present => gated bucket, no write.
- context_keys: validate_entry accepts the two sentinels for pis_cofins_profile.
- Contexto Fiscal UI: PIS/COFINS and CFOP_ENTRADA cards show the Bloco 2
cross-database evidence (5 installations, CRT 1 and 3: emission never reads
the PIS/COFINS cadastral fields anywhere) and a link to an ADMIN-facing,
anonymized dossie-for-the-accountant; entry_scope is now a dropdown.
- release/tests: decoupled the Hub's GTI_FISCAL_VERSION from the Agent's own
version (test_versions_are_coherent_across_hub_and_packages vs the new
test_agent_version_is_internally_coherent) - a Hub-only release must not
require touching, rebuilding or republishing the Agent.
- New read-only evidence: docs/runs/GTI_dryrun_universe_summary.json (527
products, Agent 2's real database), docs/runs/BLOCO2_PIS_COFINS_CROSS_DB_V1.json
(5 installations), docs/fiscal_completeness_matrix.md, docs/dossie_contador.md
(internal, real names) + central/static/docs/dossie_contador.md (public,
anonymized - no CNPJ/razao social/product names), docs/AGENT2_UPGRADE_RUNBOOK_V1.md.
- CLAUDE.md: recorded the permanent fiscal-mission rules (no AI/scraping, no
Firebird schema changes, deploy only via deploy_hub.ps1, never reenroll,
no remote access to client Agent hosts, no client data in unauthenticated
static routes).
No schema/migration change. Zero Firebird writes (new scripts never open a
write connection). Hub only - Agent code untouched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds an ADMIN-only "Contexto Fiscal" section to the Agent detail page: confirm/alter/clear the fiscal decisions
field_contract.resolve_product genuinely cannot derive from the ERP alone (icms_treatment, pis_cofins_profile,
entry_scope). Persists through the existing FiscalContextEntry table (the same one blocker_resolution already
writes operator answers into) - no new persistence model, no opaque JSON box. CSRF + RBAC + audit reuse the exact
mechanism already proven for /agents/{id}/features. Confirming or clearing flags every already-enriched product on
the installation for re-evaluation.
The list was reduced from 5 candidates to 3 by auditing real production data (Agent 2, 2026-09-27/28):
icms_rate/credit_rate and rounding were dropped - both proven NOT_APPLICABLE / ALREADY_DERIVED for this CRT=1
company from real field_contract dry-runs, not assumption. Real evidence surfaced in the UI is a dated, one-time
manual read-only measurement (disclosed in the card copy itself as such) - the Hub has no direct Firebird access,
so this is not a live query; a follow-up should pipe it through the heartbeat the same way CRT/CONTRIBUINTE_IPI
were wired.
Also found, during the same audit, that this client's NFC-e emission ignores the product's cadastral PIS/COFINS
CST/rate fields entirely (verified on the 3 real products that DO have them populated - the emitted documents never
carry the cadastral value) - documented as a strong OPEN recommendation for a future field_contract change, not
applied here, since changing canonical fiscal semantics needs accountant sign-off, not just my own inference.
1129 tests passed (tests + central/tests + agent/tests), 0 failed. Version 1.3.9 (single GTI_FISCAL_VERSION). Hub
only - Agent unchanged, no new installer this release.
REPROCESS_PRODUCTS is not in SYNC_JOB_KINDS, so it fell to the else branch of execute_job and hardcoded company=None
for _unified_fiscal - a real gap, not a missing feature: the same deterministic, fail-closed company_context.resolve()
every sync job already uses is cheap (no full-table scan, confirmed against the real database) and applies just as
well to a one-off reprocess job. Now resolves the real company the same way; falls back to None only when the
company genuinely cannot be resolved yet, never a silent guess. Does not touch RTC rule logic.
Verified end-to-end against the real production database (dry-run, zero writes) that this was the last domain still
carrying the old bug - field_contract's post-NCM fiscal completion already bypassed it entirely by reading
Installation.company_uf/crt/company_contribuinte_ipi directly (populated via the 1.3.7 heartbeat fix).
1123 tests passed (tests + central/tests + agent/tests), 0 failed. Version 1.3.8 (single GTI_FISCAL_VERSION).
field_contract.resolve_product was only exercised by its own unit tests. /v1/products/resolve now calls it for every product whose NCM
resolves, merging CFOP/CST/CSOSN/PIS-COFINS/CST_IPI/SITUACAO_TRIBUTARIA/IAT/IPPT decisions alongside CLASSIFICACAO_FISCAL/CEST/IPI - reusing
the same engine, the same FiscalContextEntry table the blocker-resolution workflow already writes to, no duplicated rules.
Closes three real transport/discovery gaps found while wiring it in:
- Agent heartbeat never sent company_crt (env-var UF only) or company_contribuinte_ipi, so Installation.company_crt stayed permanently empty
even though company_context.resolve() already proves both from C000004/EMISSOR for every coverage job. Resolved once, cached, retried only
while unresolved. Migration 0032 adds Installation.company_contribuinte_ipi (NULL=unknown, never coerced to "N").
- The Agent's enrichment payload only ever reported 3 old fields (CLASSIFICACAO_FISCAL/CEST/IPI) as "current", so the Hub could never see the
product's own CFOP/CST/CSOSN/... nor the classification columns (TIPO/PROD_COZINHA/PRODUTO_SERVICO/CODGRUPO) infer_role needs - separated
into a read-only CURRENT_REPORT_FIELDS list, the write allowlist (PRODUCT_ENRICHMENT_WRITE_V1) untouched.
- Added a targeted, product_code-scoped evidence query (firebird_reader.read_product_evidence: recipes/composed/purchases/stock, indexed WHERE,
never a full-table scan) so st_withheld/product_role can be proven for a single ad-hoc resolve, not only during a full sync job.
Verified read-only against the real database (D:\PROJETOS\GTISISTEMAS\Banco\GTI.FDB): CRT=1, CONTRIBUINTE_IPI=N, company_context.resolve()
reaches HIGH confidence; infer_role correctly resolved RESALE/OWN_PRODUCTION for 5/8 sample real products with zero operator questions.
1121 tests passed (tests + central/tests + agent/tests), 0 failed. Version 1.3.7 (single GTI_FISCAL_VERSION shared by Hub and Agent per
central/app/version.py's own design - never independently versioned).
Sandbox test: a 26 s outage exhausted the previous 15 s retry budget and failed a REPROCESS_ALL.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found by uninstall + fresh install on the clean Windows Sandbox: the old agent_state.sqlite identity conflicted with the new enrollment and the service sat idle.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Found in the clean-Windows recovery test: a single TLS handshake failure mid-job failed a whole sync.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- resolver (rules RTC-R0..R4/A1..A3/G1..G2) over the official cClassTrib table 2026-06-22 and LC 214/2025 annexes; local vs official catalog comparison
- ONE root-cause RTC blocker per product (missing/unresolved/ambiguous/conflict/official mismatch/catalog outdated/evidence missing)
- auto-apply: C000025.CCLASSTRIB only, three switches (FIREBIRD_WRITES + RTC_AUTO_APPLY_ENABLED + Hub authorization), per-product re-read and re-decision, never overwrites, rowcount 1, audit table rtc_write_audits (migration 0027)
- copy-only write E2E (280/280 applied, only CCLASSTRIB changed, 2nd run 0 writes, stale decision 0 writes); real databases untouched
- docs: cclasstrib_resolution, rtc_auto_apply, unified_fiscal_completeness
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- RTC: Agent reads ALL PARAMETROS rows every sync (conflict => off), reports rtc_source/rtc_observed_at/rtc_param_rows; legal_effective_context is informational only
- PIS/COFINS: applicability from CRT + PISCOFINS flag + incidence only (never from empty fields); Simples w/o special treatment NOT_APPLICABLE, taxed regime with explicit N OPTIONAL, unreadable declaration NEEDS_REVIEW
- CST: one digit with an official zero-padded form is VALID_AFTER_NORMALIZATION (non-blocking warning), nothing is written to the ERP
- tests: RTC toggle 0->1->0 across syncs (Agent + Hub stale closing), non-circularity, normalization, panel provenance
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- central/app/unified_fiscal_coverage.py: pure rules (PIS/COFINS NOT_APPLICABLE/REQUIRED/OPTIONAL/NEEDS_REVIEW, RTC only when the ERP switch is on)
- agent: read-only loader (unified_fiscal.py) replaces the legacy/expanded module; findings feed the single operational state, coverage and blocker queue
- hub: BLOCKED_FISCAL_DATA blockers (PIS/COFINS/RTC) share the queue; stale ones close on the next full pass; dashboard breakdown gains CEST/PIS/COFINS/RTC
- panel: legacy/expanded coverage block removed; discreet 'Reforma Tributaria: Ativa/Desativada'
- docs: unified_fiscal_completeness, pis_cofins_applicability, rtc_pis_cofins_model + official source manifests; read-only E2E script
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Hub: installations.processing_enabled / sync_interval_seconds / last_sync_* (migration 0026).
The first heartbeat of an installation without coverage queues exactly one INITIAL_SYNC;
afterwards one SYNC_INCREMENTAL per interval (idempotent, backs off on failure). One fiscal
job at a time per Agent (FIFO, stale RUNNING recovery). Job completion now updates the
coverage snapshot, blockers and JOB_STARTED/COMPLETED/FAILED events. Panel: Sincronizar
agora / Reprocessar tudo / Pausar / Retomar + status; pause is enforced Hub-side.
Agent: local AGENT_JOBS_ENABLED is ignored (central control); SYNC_INCREMENTAL re-classifies
only new/changed products via a per-product fingerprint+decision cache and rebuilds coverage
from it; results carry operational summary and blockers; Firebird credentials come from a
DPAPI firebird.bin. Setup: read-only SGV Firebird discovery (auto-select only when
unambiguous, wizard chooser otherwise), starts the service, waits for the first heartbeat,
upgrade-in-place mode. All jobs stay read-only (FIREBIRD_WRITES=0). Version 1.1.3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the native list with a combobox (name/code/CNPJ search, accent- and
case-insensitive, keyboard + mouse, custom dropdown, hidden client_id = the
Gerenciador's real CLIENTE.ID). The backend resolves client_id against the
active GTI_GERENCIADOR mirror and rejects free text/unknown ids. Adds
Tenant.external_cnpj (migration 0025) mirrored via the internal endpoint.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
--env-file only feeds the containers; a plain shell script never saw
BACKUP_DIR, so it silently fell back to ./backups while the Hub itself used
the real value from .env. Found in production: postgres's /backups bind
mount was still pinned to a directory orphaned by an earlier git-archive
swap (mv gti-fiscal gti-fiscal_old && mv gti-fiscal_new gti-fiscal — Linux
bind mounts follow the original dentry, not the path string), so every
backup.sh run had been silently landing dumps nobody could find at the
live path. Fixed by pointing BACKUP_DIR at a stable absolute path outside
the deploy tree and recreating postgres once to pick it up; verified with
a real ./backup.sh run in production.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
backup.sh now also streams the fiscal-hub container's release binaries
(dist tarball, never in Postgres) into BACKUP_DIR alongside the pg_dump,
with the same 14-backup retention; restore_releases.sh is the companion
restore. Verified with a real restore smoke test in production: the
restored dump's agent_releases row and the restored tarball's exe both
match the published sha256 exactly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Tenant gains source (LOCAL|GTI_GERENCIADOR), external_client_id, external_code,
active and synced_at (migration 0024). central/app/client_sync.py fetches from
the panel's new GET /internal/fiscal/clients (dedicated bearer token, never
MySQL credentials) and upserts Tenant rows, never touching a LOCAL row.
/agents/new now offers only active GTI_GERENCIADOR tenants, with a client-side
search box; the POST validates the same rule server-side, not just in the
dropdown. central/scripts/sync_clients_from_gerenciador.py is the operator-run
sync + reconciliation report.
GTI_FISCAL_VERSION/AGENT_VERSION bumped to 1.1.2 (release/VERSION single
source of truth), incorporating the installation_id persistence fix. Rebuilt
the EXE installer trio at 1.1.2 (scripts/build_windows_installer.ps1).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
POST /v1/agents/enroll now returns installation_id; new installs persist it
to agent.env immediately. Runtime recovery precedence: existing SQLite
installation_identity -> configured INSTALLATION_ID -> authenticated
GET /v1/agents/me/identity. Recovery fails closed on missing/malformed/
mismatched identity rather than inventing a UUID (which the Hub would then
reject as a conflicting Installation on an already-enrolled Agent). An
explicit AGENT_JOBS_ENABLED gate stops the service from claiming a job right
after the first heartbeat.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
One universal Windows Agent package per version (agent_releases: version/
platform/architecture/sha256/status DRAFT|PUBLISHED|REVOKED; the binary
lives on a local release-storage volume, never in PostgreSQL). An operator
creates a pending installation (agent_enrollments) from Agents / Servidores
-> + Nova instalação, then either downloads it directly or generates a
one-time HTTPS link (short TTL, single use). Either path streams the
package plus a freshly minted enrollment.json whose token is never written
to storage in plaintext, only its SHA-256 hash. On the client host,
gti-agent.ps1 install-auto verifies the bundle's SHA-256, refuses a non-
HTTPS Hub, enrolls via POST /v1/agents/enroll (deleting the enrollment
token immediately after, success or failure), writes an ACL-restricted
.env with the newly issued permanent Agent token, and registers the
Windows Service without starting it (Firebird stays unconfigured on
purpose). Idempotent, replay-proof, rate-limited per IP, fully audited,
never logs a secret.
Migration 0022_agent_releases_enrollment (verified fresh-DB ==
upgrade-from-0021 by pg_dump diff). GTI_FISCAL_VERSION/AGENT_VERSION
bumped to 1.1.0 as the single source of truth across code, packages, the
release manifest and every published AgentRelease row.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Collaborators logged into GTI Gerenciador open GTI Fiscal without a
second password. The panel signs a 60 s JWT (Ed25519; the Hub holds only
the public key); the Hub checks signature, iss/aud/exp/nbf, maximum
lifetime, nonce against a one-time state cookie set when the flow starts
at GET /login (login-CSRF protection) and a single-use jti
(sso_used_assertions). Collaborators are provisioned in admin_users with
auth_source=GTI_GERENCIADOR and no password; a CHECK constraint makes a
passwordless local account or a federated account with a password
impossible (migration 0021). Sessions last FEDERATED_SESSION_SECONDS
(1 h) and are renewed silently through the panel.
The local admin stays as break-glass at /login/local, audited. Logout ends
only the Hub session. Every SSO success/rejection (stable reason code),
logout and local login is audited with the real client IP, never the
token. verify_password now fails closed on a missing hash instead of
raising.
Visual identity: gti-theme.css is a verbatim copy of the panel's
design system (tokens, sidebar, light/dark theme) plus its tema.js,
sidebar.js and logo; control-panel.css keeps only Hub components, all
colours from the panel tokens. Page structure and features unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GTI terminates TLS in its Nginx Proxy Manager (10.0.10.100), not on the
Hub host, so the Hub must be reachable on the LAN — by the proxy only.
- docker-compose.prod.yml passes FORWARDED_ALLOW_IPS (uvicorn then trusts
X-Forwarded-For/-Proto from the proxy only: real client IPs in audit and
session records, https scheme) and WEB_COOKIE_SECURE. Defaults keep the
previous behaviour.
- restrict_hub_port.sh + gti-fiscal-hub-port.service: Docker-published
ports bypass UFW, so the source restriction lives in DOCKER-USER,
matched on the conntrack original destination with --ctdir ORIGINAL,
and is re-applied whenever docker.service starts (the chain is not
persisted across reboots).
- *.service forced to LF like *.sh; tests cover both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RELEASE_MANIFEST.json still said head 0011 (repo was at 0018, now 0020)
and RELEASE_CANDIDATE_V1.md said 0010. Updated both, plus
AI_PROJECT_STATE.md and PILOT_DEPLOYMENT.md, to head
0020_not_null_timestamps as applied in production.
scripts/release_manifest.py reported the 4-digit file prefix as the head
("0020" instead of "0020_not_null_timestamps") and always read the
applied revision from the dev container. It now asks Alembic for the real
head and revision count, and the Hub container is configurable
(HUB_CONTAINER); the source is recorded in the manifest.
tests/test_release_candidate.py now fails when the manifest head differs
from Alembic's head.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Audit of models vs fresh DB vs production (all at 0019): 20 columns are
NOT NULL in the models but nullable in the database, all timestamps with
DEFAULT now() created by 0011/0012/0014/0017 without nullable=False.
Classified DB_CONSTRAINT_MISSING: no application path writes NULL to
them (inserts omit them, updates set now; no raw SQL inserts), and there
were zero NULLs in production and in the development database (246,682
rows checked). The upgrade aborts without changing anything if any NULL
is present, instead of inventing a value.
After this revision a fresh database matches the models on nullability,
column types, PKs, FKs, uniqueness and checks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found by running deploy/hub/backup.sh on the production VM:
- With core.autocrlf=true, `git archive` on the Windows release machine
shipped the .sh files with CRLF and `sh` aborted on line 4
("set: Illegal option -"): backup, restore and seed never ran.
.gitattributes now forces *.sh to LF.
- Docker creates the bind-mounted backups/ directory as root, so the
host-side retention `rm` would fail once the 15th dump existed.
Retention now runs inside the postgres container.
tests/test_deploy_scripts.py covers both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0001 called Base.metadata.create_all(), building the schema from the
current models, so on a fresh PostgreSQL 0002 collided on
datasets.raw_sha256 and `alembic upgrade head` could never run from zero.
0001 is now an explicit, frozen baseline: the seven base tables exactly as
the real history created them, before 0002/0006 extended them. Databases
already past 0001 never re-run it. Proof: a fresh `upgrade head` produces a
schema identical (pg_dump --schema-only) to the long-lived dev database.
0019 repairs a latent bug found by that comparison: 0006 used
server_default="now()" as a string, which PostgreSQL froze at the instant
the column was created, so every Agent and Job got the same created_at.
It also adds three indexes the models declare but no migration created.
central/tests/test_migration_baseline.py guards (via AST) against
migrations importing the models or using string timestamp defaults.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gtiandClaude Sonnet 5