chore: establish project baseline
This commit is contained in:
OpenCode
@@ -0,0 +1,231 @@
|
||||
# Mission 00D Structured Firebird Boundary Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace arbitrary SQL execution with a structured read-only Firebird API, pass the pre-connection security gate, then connect only to the authorized database and complete discovery.
|
||||
|
||||
**Architecture:** A private executor selects static catalog queries by operation key and is the only layer allowed to call the Firebird cursor. `StructuredFirebirdReader` exposes fixed catalog operations and a catalog-issued identity for safe sampling. The old SQL parser remains offline-only or is removed from runtime authority; the live path never accepts consumer SQL.
|
||||
|
||||
**Tech Stack:** Python 3.10+, existing `fdb` 2.0.4 and installed Firebird 2.5.9 client, pytest, existing collector/fingerprint/mapping modules.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Only `D:\PROJETOS\GTISISTEMAS\Projeto\TESTESOFICIAIS.FDB` may be opened or queried.
|
||||
- Files outside `Projeto` are `READ_ONLY_EXTERNAL_SYSTEM` and must not be modified.
|
||||
- `OTHER_DATABASES_ACCESSED = 0`.
|
||||
- `APPLICATION_DDL_DML_WRITES = 0`.
|
||||
- The public adapter must not expose `execute(sql)`, `query(sql)`, `raw_execute(sql)`, `execute_statement(sql)`, or equivalents.
|
||||
- Procedures and UDFs may be inventoried as metadata but never executed.
|
||||
- Credentials, raw BLOBs, full rows, customer data, original ERP database paths, and secrets must not enter reports or source.
|
||||
- No Firebird installation, DLL, service, registry, permission, or configuration changes.
|
||||
- No real database access before the security gate in Task 4 passes.
|
||||
- Git identity must not be invented or configured; report `GIT_IDENTITY_NOT_CONFIGURED` when absent.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Replace the raw execution boundary with a private query executor
|
||||
|
||||
**Files:**
|
||||
- Create: `src/gti_discovery/firebird_queries.py`
|
||||
- Modify: `src/gti_discovery/read_only.py`
|
||||
- Modify: `src/gti_discovery/__init__.py`
|
||||
- Test: `tests/test_read_only_boundary.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces private `_execute_catalog(adapter, operation_key, params=()) -> Any`.
|
||||
- Produces immutable internal query mapping keyed by approved operation names, never by caller-provided SQL.
|
||||
- Public package exports no raw SQL execution function or `ReadOnlyConnection.execute` method.
|
||||
|
||||
- [ ] **Step 1: Write failing API-boundary tests**
|
||||
|
||||
```python
|
||||
def test_public_connection_has_no_sql_execute_method():
|
||||
connection = make_structured_connection()
|
||||
assert not hasattr(connection, "execute")
|
||||
assert not hasattr(connection, "query")
|
||||
assert not hasattr(connection, "raw_execute")
|
||||
|
||||
def test_unknown_operation_key_never_reaches_driver():
|
||||
adapter = RecordingAdapter()
|
||||
with pytest.raises(ValueError):
|
||||
_execute_catalog(adapter, "SELECT * FROM X")
|
||||
assert adapter.calls == []
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused tests and verify failure**
|
||||
|
||||
Run `python -m pytest tests/test_read_only_boundary.py -q`. Expected: FAIL because the current public wrapper exposes `execute` and the private operation executor does not exist.
|
||||
|
||||
- [ ] **Step 3: Define static internal query operations**
|
||||
|
||||
Move catalog SQL into `firebird_queries.py` under fixed keys such as
|
||||
`database`, `relations`, `fields`, `keys`, `indexes`, `foreign_keys`,
|
||||
`triggers`, `procedures`, `procedure_parameters`, `packages`, `generators`,
|
||||
`dependencies`, and `monitoring`. The executor accepts only one of these
|
||||
keys, binds values supplied by the operation implementation, and calls the
|
||||
private driver cursor. It must not accept SQL text, fragments, clauses, or
|
||||
callable expressions.
|
||||
|
||||
- [ ] **Step 4: Remove runtime authority from `ReadOnlyConnection.execute`**
|
||||
|
||||
Replace the public wrapper with a private connection implementation that is
|
||||
constructed only by the structured reader. Keep read-only transaction setup,
|
||||
adapter capability checks, close behavior, and fail-closed cleanup. Remove
|
||||
`validate_read_only` and raw execution from package exports if no runtime
|
||||
consumer depends on them; retain the parser only as an explicitly offline
|
||||
diagnostic utility if existing tests need it.
|
||||
|
||||
- [ ] **Step 5: Run boundary and existing safety tests**
|
||||
|
||||
Run `python -m pytest tests/test_read_only_boundary.py tests/test_read_only.py -q`. Expected: PASS after updating tests to assert the new architecture rather than raw SQL execution.
|
||||
|
||||
- [ ] **Step 6: Record the root cause**
|
||||
|
||||
Add a report stating that quoted identifiers bypassed lexical callable detection and that the incomplete allowlist rejected `CAST`/aggregates. State that the parser is no longer a runtime security boundary.
|
||||
|
||||
### Task 2: Implement the structured catalog reader
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/gti_discovery/collector.py`
|
||||
- Modify: `src/gti_discovery/firebird_connection.py`
|
||||
- Test: `tests/test_collector.py`
|
||||
- Test: `tests/test_firebird_connection.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces `StructuredFirebirdReader.get_database_info()`.
|
||||
- Produces `list_relations()`, `get_relation_columns(relation)`, `get_primary_keys(relation)`, `get_foreign_keys(relation)`, `get_indices(relation)`, `get_triggers()`, `get_procedures()`, `get_generators()`, `get_dependencies()`, and optional monitoring methods.
|
||||
- `collect_metadata(reader)` accepts only the approved reader runtime type and calls structured methods, never SQL.
|
||||
|
||||
- [ ] **Step 1: Write failing structured-reader tests**
|
||||
|
||||
Use a fake private executor keyed by operation name and assert each public method requests the expected key. Assert an arbitrary adapter, fake object with matching method names, and raw SQL string are rejected by `collect_metadata`.
|
||||
|
||||
- [ ] **Step 2: Implement fixed catalog methods**
|
||||
|
||||
Each method calls the private operation executor with a fixed key. Metadata queries may contain `CAST`, `COUNT`, `SUM`, `MIN`, `MAX`, `COALESCE`, joins, and grouping. No query is assembled from caller text. Optional catalog failures become `unavailable`; permission failures remain `query_error`.
|
||||
|
||||
- [ ] **Step 3: Preserve sanitized metadata**
|
||||
|
||||
Keep field names, relation names, types, nullability, positions, keys, indexes, trigger timing/type, procedure metadata, generator names, and dependency names. Redact binary/source/default/sensitive values and never select routine source or BLOB contents.
|
||||
|
||||
- [ ] **Step 4: Update collection and fingerprint fixtures**
|
||||
|
||||
Adapt collector tests to the structured reader and assert catalog-shaped rows retain safe fields. Assert fingerprint ordering, positional fields, optional capabilities, and `SCHEMA_UNKNOWN` for incomplete metadata remain correct.
|
||||
|
||||
- [ ] **Step 5: Run the focused collector suite**
|
||||
|
||||
Run `python -m pytest tests/test_collector.py tests/test_fingerprint.py tests/test_read_only_boundary.py -q`. Expected: PASS with no database access.
|
||||
|
||||
### Task 3: Add catalog-issued safe sampling
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/gti_discovery/collector.py`
|
||||
- Create: `src/gti_discovery/identifiers.py`
|
||||
- Test: `tests/test_sampling.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces `quote_firebird_identifier(name) -> str` for validated identifiers.
|
||||
- Produces `StructuredFirebirdReader.sample_relation_rows(relation, columns, limit) -> list[dict]`.
|
||||
- Sampling requires relation and columns previously issued by `list_relations`/`get_relation_columns`; it accepts no SQL, WHERE, ORDER BY, expression, procedure, or UDF.
|
||||
|
||||
- [ ] **Step 1: Write failing sampling tests**
|
||||
|
||||
Cover valid catalog-issued relation/column identities, quoted names, injection strings, unknown relation/column, procedure-like relation, expression input, negative/oversized/non-integer limit, and raw BLOB redaction.
|
||||
|
||||
- [ ] **Step 2: Implement identifier validation and quoting**
|
||||
|
||||
Require exact membership in the reader’s catalog snapshot, reject control characters, dots, parentheses, semicolons, comments, and expressions, and quote embedded double quotes using Firebird identifier rules. Preserve the original catalog identity for comparison; never normalize arbitrary text into an accepted identifier.
|
||||
|
||||
- [ ] **Step 3: Implement bounded sampling**
|
||||
|
||||
Generate the internal sample operation from validated identifiers and a validated limit capped at a small constant. The private executor receives an operation key plus structured identifier parameters; it never receives a complete SQL string from the caller.
|
||||
|
||||
- [ ] **Step 4: Sanitize sample output**
|
||||
|
||||
Return only permitted scalar metadata evidence. Redact binary values, sensitive names, oversized values, and customer/commercial values not needed for mapping. Never execute selectable procedures or functions.
|
||||
|
||||
- [ ] **Step 5: Run sampling tests**
|
||||
|
||||
Run `python -m pytest tests/test_sampling.py tests/test_collector.py -q`. Expected: PASS with all injection and quoted-identifier cases rejected structurally.
|
||||
|
||||
### Task 4: Pass the pre-connection security gate
|
||||
|
||||
**Files:**
|
||||
- Modify: `tests/test_read_only.py` or replace with offline parser tests
|
||||
- Modify: `tests/test_firebird_discovery_script.py`
|
||||
- Modify: `docs/MISSION_00_REPORT.md`
|
||||
- Create: `.superpowers/sdd/2026-09-16-firebird-mission-00d/task-4-report.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces a reproducible gate result proving no public arbitrary SQL reaches the driver.
|
||||
- Produces no database access.
|
||||
|
||||
- [ ] **Step 1: Add regression tests for parser non-authority**
|
||||
|
||||
Keep offline tests for quoted identifiers, single/double quotes, comments, semicolons, `EXECUTE PROCEDURE`, selectable procedures, UDFs, `UNION`, subqueries, malicious aliases, reserved words in strings, valid quoted names, and valid relation/column names. Assert these tests exercise only the offline helper and cannot invoke the private executor.
|
||||
|
||||
- [ ] **Step 2: Add internal Firebird expression fixtures**
|
||||
|
||||
Test static internal query text containing `CAST`, `COUNT`, `SUM`, `MIN`, `MAX`, `COALESCE`, and `GROUP BY` through the operation-key executor fixture. Assert no blacklist parser is called and the driver receives only known internal operations.
|
||||
|
||||
- [ ] **Step 3: Run the gate commands**
|
||||
|
||||
Run:
|
||||
|
||||
```text
|
||||
python -m pytest tests/test_read_only_boundary.py tests/test_sampling.py tests/test_collector.py tests/test_firebird_connection.py tests/test_firebird_discovery_script.py -q
|
||||
python -m pytest tests -q
|
||||
```
|
||||
|
||||
Expected: all pass, with no database connection attempt. If any gate condition fails, stop with `MISSION_00_BLOCKED`.
|
||||
|
||||
- [ ] **Step 4: Update the implementation report**
|
||||
|
||||
Document the actual public methods, private executor, transaction read-only behavior, parser role, and gate result. Do not claim live connectivity yet.
|
||||
|
||||
### Task 5: Connect and complete real discovery only after the gate
|
||||
|
||||
**Files:**
|
||||
- Modify: `scripts/run_firebird_discovery.py`
|
||||
- Modify: `src/gti_discovery/mapping.py`
|
||||
- Modify: `docs/AI_PROJECT_STATE.md`
|
||||
- Modify: `docs/DATABASE_DISCOVERY.md`
|
||||
- Modify: `docs/SCHEMA_MAPPING.md`
|
||||
- Modify: `docs/MISSION_00_REPORT.md`
|
||||
- Create: `.superpowers/sdd/2026-09-16-firebird-mission-00d/task-5-report.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes the structured reader and safe sampling API from Tasks 1-4.
|
||||
- Produces sanitized engine facts, catalog metadata, UI-to-column mappings, image/incremental/trigger evidence, and real fingerprint only if connection succeeds.
|
||||
|
||||
- [ ] **Step 1: Revalidate installed Firebird read-only**
|
||||
|
||||
Inspect services, processes, client DLL, tools, and configuration under external ERP paths without changing them. Use the already identified Firebird 2.5.9 client only; do not install or start anything.
|
||||
|
||||
- [ ] **Step 2: Connect only to the authorized target**
|
||||
|
||||
Require the pre-connection gate result, validate the absolute target, require credentials from the process environment without logging them, configure Firebird read-only isolation, issue the minimal identity query first, and compare the returned identity with the authorized target. On any failure, stop with `MISSION_00_BLOCKED`.
|
||||
|
||||
- [ ] **Step 3: Execute structured discovery**
|
||||
|
||||
Collect database facts and catalogs through fixed operations. Use only catalog-issued sampling to investigate candidate products, stock, fiscal concepts from the ERP UI (`NCM`, `CFOP`, `CST ICMS`, `ICMS rate`, `CEST`, `CSOSN`, PIS/COFINS, IPI, origin, IAT, incidence, IPPT), image/photo tab structures, timestamps/audit/generators, and trigger/procedure dependencies. Mark each mapping `CONFIRMED`, `INFERRED`, or `UNKNOWN`.
|
||||
|
||||
- [ ] **Step 4: Generate fingerprint V1**
|
||||
|
||||
Hash sanitized engine context, product structure, fiscal fields, relationships, image structure, trigger metadata, and relevant positional catalog data. Treat incomplete or ambiguous collection as `SCHEMA_UNKNOWN`.
|
||||
|
||||
- [ ] **Step 5: Update final documents**
|
||||
|
||||
Replace stale blocked claims only when live evidence exists. Otherwise preserve `MISSION_00_BLOCKED` and record the exact blocker. Include all safety invariants and no future write/API/service/scheduler proposal.
|
||||
|
||||
- [ ] **Step 6: Run final verification**
|
||||
|
||||
Run `python -m pytest tests -q`, `git diff --check`, `git status --short --branch`, and Git identity checks. Confirm no other `.FDB` was accessed, no external file changed, no secret exists in project text, and no DDL/DML path exists.
|
||||
|
||||
## Plan Self-Review
|
||||
|
||||
- The root-cause, structured boundary, static query ownership, safe sampling, procedure/UDF exclusion, parser demotion, defense in depth, gate, live connection, discovery, fingerprint, documentation, and Git requirements from the specification map to Tasks 1-5.
|
||||
- No task permits database access before Task 4 passes.
|
||||
- Later interfaces use exact names introduced by earlier tasks.
|
||||
- The old parser is never used to authorize internal Firebird query syntax.
|
||||
- No placeholder, unbounded sampling, arbitrary SQL input, or external file modification is permitted.
|
||||
Reference in New Issue
Block a user