2GT-RichandClaude Opus 5
273a0da737
Closes #121. The Quick Deploy file was missing NETFLOW_BIND_PORT and the 2055/udp mapping, so the collector bound inside the container and nothing reached it. NetFlow did nothing, silently, with an empty traffic page and no error — for everyone following the path the README tells new users to take. The report was accurate and understated. docker-compose.ghcr.yml had drifted for seven weeks, and three closed issues had never reached it: * NETFLOW_BIND_PORT and the UDP port — as reported * all five POLLER_* tunables — #114, whose fix was "pass the tunables through" * the Redis memory ceiling and eviction policy — #116 * TZ on the postgres service, which nobody had noticed at all The middle one is the uncomfortable part. Four days ago #114 was closed by forwarding poller settings that were documented but inert, with a comment saying setting them "did precisely nothing". On Quick Deploy that remained true, because one compose file was fixed and the other was not — including the advice given to that reporter about which knob to turn. So patching the four gaps would have been treating the symptom. The two files describe the same system and differ only in where images come from; maintaining both by hand meant remembering every change twice, and the one forgotten was always the file no maintainer runs. docker-compose.ghcr.yml is now generated from docker-compose.yml by scripts/generate-ghcr-compose.mjs, which performs the two build-to-image swaps and nothing else. Substitution is textual rather than parse-and-reserialise so comments survive — the reasoning about Redis eviction is worth more than the two lines it explains. The file stays committed and browsable, because the README tells people to curl it directly. A --check mode runs as the first preflight gate and as its own CI job, both of which fail when the committed file does not match what the generator produces. Verified by deleting the NetFlow line and watching the guard reject it, with the reason and the fix in the failure message. Reported by @mamb1329, on their first issue. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
6.7 KiB
Architecture
For contributors, and for anyone wanting to know what happens between the UI and a device.
The shape of it
Browser ──► nginx ──► Express API ──► RouterOS API / SSH ──► devices
│
├──► PostgreSQL current state, config, users, audit
├──► InfluxDB time-series metrics
└──► Redis job queue and locks
Three data stores, because they answer different questions:
| Store | Holds | Why not the others |
|---|---|---|
| PostgreSQL | Devices, interfaces, VLANs, clients, users, audit log, findings | Relational, needs joins and constraints |
| InfluxDB | CPU, memory, traffic, client counts, RF quality | Time-series at poll cadence; would bloat Postgres |
| Redis | BullMQ job queues, per-device locks, poll gating | Ephemeral, needs atomic operations |
Polling
A scheduler enqueues work per device on separate cadences, so a slow operation never delays a fast one:
| Cycle | Default | Collects |
|---|---|---|
| Fast | 30s | Reachability, CPU, memory, traffic counters, clients |
| Slow | 5min | Interfaces, VLANs, addresses, neighbours, STP, wireless, CAPsMAN |
| Logs | 60s | Device log entries, feeding log alerts |
| Config snapshot | 60min | /export, deduplicated by content hash |
| Config health | 60min | Read-only configuration audit |
| MAC / spectral / AP scan | configurable | Switch MAC tables, RF surveys |
Devices are probed once for wireless capability and the result recorded, so non-wireless devices are skipped cheaply on every later poll.
Device communication
RouterOSClient speaks the RouterOS binary API (8728, or 8729 with TLS). It is used
for essentially everything, including the safety machinery, because many deployments never
configure SSH.
SSH is used only where RouterOS offers no API equivalent — /export for config snapshots,
binary backup retrieval, packet capture download, and the in-browser terminal.
Where the interesting logic lives
backend/src/services/
├── mikrotik/
│ ├── RouterOSClient.ts binary protocol client
│ ├── DeviceCollector.ts all collection and device mutation
│ └── capsman.ts role detection, radio matching (pure)
├── changeGuard/
│ ├── ChangeGuard.ts restore point, scheduler, verify, commit
│ ├── pathModel.ts live snapshot, management-path resolution (pure)
│ ├── invariants.ts what must hold for management to survive (pure)
│ ├── analyzeChange.ts simulate a change, diff the invariants (pure)
│ └── configHealth.ts standing audit rules (pure)
├── topology/
│ └── buildTopology.ts neighbour graph construction (pure)
├── netflow/ NetFlow v9 / IPFIX collector and aggregator
└── PollerService.ts scheduling and workers
Anything marked pure is a function over a snapshot with no I/O. That is deliberate: this logic is subtle, its failure mode is a plausible-looking wrong answer rather than a crash, and it must be testable without the hardware it reasons about. Those modules carry the bulk of the test suite.
Repository layout
mikrotik-manager/
├── frontend/ # React + TypeScript (Vite)
│ └── src/
│ ├── pages/ # One component per route
│ ├── components/ # Shared UI
│ ├── services/ # API client (Axios)
│ ├── hooks/ # Custom React hooks
│ └── types/ # TypeScript definitions
│
├── backend/ # Node.js + Express + TypeScript
│ └── src/
│ ├── routes/ # REST API handlers
│ ├── services/ # See above
│ ├── db/ # Schema and migrations
│ ├── config/ # Database, InfluxDB, Redis connections
│ ├── middleware/ # Auth, audit logging, error handling
│ └── utils/ # Crypto, OUI lookup, VLAN parsing
│
├── docs/ # This documentation
├── nginx/ # Reverse proxy config and Dockerfile
├── scripts/ # CI preflight and tooling
├── docker-compose.yml # Build from source
├── docker-compose.ghcr.yml # Pre-built images — GENERATED, do not edit by hand
└── .env.example
Database migrations
A single idempotent SQL script in backend/src/db/migrate.ts runs on every backend start.
Statements are written to be safe to re-run — CREATE TABLE IF NOT EXISTS,
ADD COLUMN IF NOT EXISTS, and guarded constraint swaps — so upgrading is just starting
the new image.
Developing against hardware you do not have
Two bugs shipped in v0.23.5 and v0.23.6 because CAPsMAN code was written and
"verified" against a fleet containing no CAPsMAN. Neither was catchable by unit
tests: both lived in SQL — a join that could never match, and a WHERE clause that
filtered out every managed radio before the analysis ran.
scripts/seed-capsman-fixture.sql reproduces the shape of a real deployment so
those paths can be exercised locally against the actual endpoints:
docker compose exec -T postgres psql -U mikrotik -d mikrotik_manager \
-q -f - < scripts/seed-capsman-fixture.sql
# ... work, hit /api/wireless/rf/channels, /api/wireless/capsman ...
docker compose exec -T postgres psql -U mikrotik -d mikrotik_manager \
-q -f - < scripts/seed-capsman-fixture-clean.sql
The fixture is deliberately awkward in the ways real deployments are:
- the controller is a router with no radios of its own, so anything keyed on
device_type = 'wireless_ap'misses it - CAP interfaces carry NULL frequency, SSID and band, because the controller owns the configuration — code that requires a frequency silently drops them
- the controller mirrors every CAP interface locally, so a CAP's radio MAC legitimately appears on two devices
- channels appear only in RouterOS's
<freq>/<phy>/<control-positions>form, never as a plain number - the radios are arranged to produce one partial overlap and one co-channel pair, so interference classification has something real to get wrong
Everything it creates is prefixed fixture-. Add to it rather than working from
assumptions whenever a report arrives with device output attached — that output is
the most valuable thing in the issue.
Before you push
scripts/ci-preflight.sh runs the same gates as CI — lint, type-check, build, tests and
production audits for both packages — in about 35 seconds. Running it locally is faster
than finding out from a red badge.