v0.23.17 stopped the poller creating an unbounded backlog but did nothing about
the one already there. A reporter with sixty devices upgraded and still had
752,318 jobs queued, nine stale devices, and an endpoint cheerfully reporting
1.59x headroom. Fair to call that not fixed.
Three faults, all mine.
**Retention never touched pending work.** cleanupJobHistory cleaned 'completed'
and 'failed' only. The queue of outstanding polls — the part the operator
actually feels — was out of scope, so an upgrade left it draining for days.
Stale pending jobs are now swept, at startup and every five minutes. The rule is
that a periodic poll which has waited longer than its own cadence is worthless:
running it yields a reading the next cycle would have taken anyway, while
occupying a worker the live fleet is waiting on. Discarding queued work is safe
precisely because it is periodic — the schedule re-enqueues within one interval.
**Deduplication failed in the case it existed for.** The in-flight claim was a
Redis key with a 135-second TTL. Under a large backlog a job waits hours, so the
claim lapsed long before it ran and the scheduler re-enqueued every cycle: the
guard worked only on a queue that was already short.
Replaced with a deterministic job id, which cannot lapse. While a job for a
device and kind exists in any state, adding it again is a no-op. That requires
finished jobs to be removed immediately or the id would stay taken and the
device would never be polled again — which costs nothing now that
device_poll_stats records every attempt, duration and error.
**The headroom metric lied.** It describes flow and is blind to accumulated
work, so it read "healthy" beside three-quarters of a million queued jobs. It
now reports backlog and drain ETA alongside, plus a single status verdict that
cannot say ok while a backlog exists. Verified by injecting 15,125 jobs: status
went to 'draining' with headroom still at 85x, which is exactly the case the old
metric got wrong.
Also added, because the reporter asked and was right to: an operator-triggered
drain (POST /api/system/poller/drain), and a Polling tab in Settings showing
queue depth, headroom, backlog, per-device staleness and the drain control. It
was all buried in an API response before. A dashboard insight fires when the
poller falls behind, so a starved poller announces itself rather than presenting
as a fleet of broken devices.
Two corrections found by testing rather than reasoning:
* BullMQ rejects ':' in a custom job id. `fast:2` threw on every enqueue and
killed the scheduler outright — three of four devices went unpolled for
eight minutes before it was caught. Hyphens now, and the dedup is exercised
directly: 500 enqueues of one device yield one queued job.
* Redis maxmemory-policy moves from volatile-lru to noeviction. I had reasoned
that only TTL-bearing keys are evicted and BullMQ job hashes carry none —
true, but its *control* keys do. Job locks and stalled-check markers both
expire, and evicting one causes duplicate processing or a stalled-job storm.
BullMQ warns about this on connect and was right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Follow-ups on #97 and #106, both reported against the releases they shipped in.
#97 — managed radios were missing from Channel Usage entirely
The query required wireless_interfaces.frequency > 0. A CAPsMAN-managed access
point stores no frequency of its own: the controller owns the channel, and the
AP's own rows carry nulls. Every managed radio was therefore filtered out before
analysis ran, which is why the reporter saw all access points collapse into
"Standalone" and a controller vanish from the selector — the group had no
surviving radios to name it.
The channel now comes from capsman_radios.current_channel where present, parsed
from RouterOS's `<freq>/<phy>/<control-positions>` form. The control letters give
a precise operating width — Ce is 40 MHz, Ceee is 80 — rather than the capability
list a width field reports. Client counts prefer the controller's per-radio figure
for the same reason.
Verified only by unit test here, since we run no CAPsMAN. The local fleet still
reports its two standalone radios unchanged.
#106 — the setting had no way to change it
maps_enabled shipped without a control, which made it a setting in name only.
Added under Settings, with the reasoning stated: tiles and geocoding are outbound
requests to a third party, and the address entered is sent to them.
Also removed the "map display is off" placeholder. The reporter pointed out it can
never appear in the environment it was written for — with maps disabled from the
start no coordinates are ever geocoded, so the block was dead code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes#93.
Routing push notifications through email ingestion was a workaround. ntfy is now
a first-class channel alongside Slack, Discord, Telegram and email, using the
same alert_channels row and the same dispatch switch.
Built to cover the whole plausible configuration space rather than waiting on the
reporter's answers, so no config migration is needed once they arrive:
- server_url is operator-supplied and defaults to https://ntfy.sh, so a
self-hosted instance works — the likely case for infrastructure alerting
- both of ntfy's auth schemes: an access token (preferred; scopable and
revocable) or username/password, with none required for an open topic
- optional click_url deep-links each notification to the device it refers to
Published as JSON to the server root rather than POST /{topic} with header
metadata: the header form is ASCII-only and these titles carry per-event emoji.
Severity maps onto ntfy's priority scale so the channel stays worth being woken
by — 4 (breaks through do-not-disturb) for device_offline, log_error and
CPU/memory pressure; 2 for recoveries and discoveries; 3 for everything else.
The event type is attached as a tag so a client can be filtered to wake only for
chosen events without needing a channel per event.
Secrets follow the existing pattern: token and password are added to
SENSITIVE_KEYS, so they are masked on read and the mask placeholder is restored
to the stored value on write rather than overwriting it.
The type column is a CHECK constraint rather than a lookup table, so widening it
means dropping and recreating the constraint; done idempotently for existing
installs.
Note on trust: server_url lets an admin point the channel at an internal address.
That is the same exposure the existing Slack and Discord webhook URLs already
have, and only an admin can configure any of them. Restricting it would break the
self-hosted case this feature is mostly for.
Verified against the real ntfy.sh — a test alert arrived with the correct title,
emoji, body, priority, tag and click URL; masking verified to hide the token on
read while preserving it across a save that did not retype it. 13 unit tests
cover priority mapping, both auth schemes, URL normalisation, and rejection of
a missing topic or non-http server.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolves#86 — MikroTik Manager was attempting SSH connections to devices that
have no SSH credentials configured, flooding RouterOS logs with failed logins.
Cause: config snapshots (a separate feature from scheduled backups, enabled by
default, hourly) capture `/export` over SSH via BackupService.exportConfig, which
silently falls back to the API credentials for the SSH login. On devices where
SSH was never set up, that auth fails every interval (and on each restart), and
there was no UI switch to turn it off.
- PollerService.processConfigSnapJob now skips devices without SSH credentials,
so no SSH connection is attempted where SSH isn't configured. Manual "Capture
snapshot" / backup actions are unchanged (user-initiated, with error feedback).
- Add a "Configuration Snapshots" section to Settings → General: enable toggle,
interval, and retention (the feature was on-by-default with no UI control),
with a note that it runs over SSH and only on devices that have SSH creds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address the actionable findings from an adversarial review:
- 2FA enrollment now requires the current password (/totp/setup), so a
hijacked session or XSS can't enable TOTP and lock the real user out.
SSO-only accounts (no local password) are told to set one first. Frontend
collects the password before starting setup.
- OIDC config rejects default_role='admin': auto-provisioning unmatched users
straight to admin is almost never intended — admin must come from an explicit
group → role mapping.
- Rate limiter no longer fails fully open on a Redis outage. Redis calls are
bounded by an 800ms timeout (the client uses maxRetriesPerRequest:null, so
commands would otherwise hang, not throw), and on failure a per-process
in-memory sliding-window fallback keeps /login and other limited endpoints
throttled. Verified with Redis stopped: attempts throttle (429) instead of
hanging or passing unlimited.
Findings intentionally not actioned: the "API-token auth is fire-and-forget"
report was a false positive (Express awaits next(), which is called inside the
promise), and the webhook DB-pool and JWT-revocation items are either
overstated or a separate architectural decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add OpenID Connect single sign-on so the platform can authenticate against
any standards-compliant provider (Entra ID, Okta, Google Workspace, Keycloak,
Authentik, …) alongside local accounts. Resolves the feature request in #73.
Flow (backend-driven Authorization Code + PKCE):
- GET /api/auth/oidc/login builds the authorize URL and stores the PKCE/nonce
transaction in Redis (single-use, 10m TTL); the callback validates the ID
token via openid-client (JWKS signature, iss/aud/exp/nonce), resolves the
identity to a local user, and mints the platform's normal session JWT —
handed to the SPA via URL fragment. No sessions/cookies introduced.
Identity mapping (users table gains email/oidc_subject/oidc_issuer/
auth_provider; password_hash now nullable; partial unique indexes):
- First sign-in auto-provisions a user, or links to an existing local account
when the IdP email is verified and matches.
- Roles derive from a configurable IdP group -> role map with a default
fallback; existing users are never auto-demoted on a groupless login.
Configuration (admin-only, no env/redeploy needed):
- Stored as an app_settings row with the client secret encrypted at rest
(utils/crypto); masked in responses and excluded from GET /api/settings.
- New UI: Settings -> SSO / OIDC panel (issuer/client, claim + group->role
mapping, provisioning toggles, discovery test, copyable redirect URI), a
login-page SSO button, and an SSO badge in Users & Roles.
Security: local break-glass login preserved; PKCE + single-use state; email
linking requires email_verified; same-site-only post-login redirect.
Adds openid-client@^5 (CommonJS). 17 new unit tests (claim/role mapping,
username derivation, config secret round-trip/masking); 114 backend tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
UI/UX consolidation pass to reduce nav sprawl (~24 destinations -> ~17):
- Merge Router Settings + Switch Settings (copy-paste duplicates) into one
Network Services > Discovery & SNMP page with an All/Routers/Switches
scope selector; LLDP + SNMP status tables merge both fleets with a
type pill, apply fans out per scope
- Retire Routers/Switches overview pages; the Devices type filter now
lives in the URL (?type=AP|SW|RTR) and legacy routes redirect to the
pre-filtered view. Wireless overview kept (RF Health / Rogue APs are
unique content)
- Fold Wireless Clients into Clients: All/Wired/Wireless segmented
filter (URL-synced); wireless view swaps the Type column for SSID +
signal quality columns
- Regroup sidebar into Monitor / Network / Services / Operations
sections; Devices and Wireless are collapsible groups (auto-open on
active section, manual collapse always wins); Wireless group renders
only when wireless APs exist
- Rename Syslog -> Logging with a cross-link to live Events; group the
Settings tabs into Platform / My Account / Fleet sections
- Legacy routes (/routers, /switches, /routers/settings,
/switches/settings, /wireless/clients) redirect so bookmarks keep
working
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Guest WiFi (Hotspot): one-click captive-portal wizard that creates the guest
SSID (virtual AP per radio), VLAN/bridge segregation + L3 VLAN interface, pool,
DHCP, portal, rate-limited guest profile, and masquerade; vouchers with limits +
printable sheets; live guest sessions; walled garden.
Network intelligence: per-client device fingerprinting (vendor/hostname →
category) with a persisted per-client override on the detail page; rogue/neighbor
AP detection from stored AP scans (evil-twin flagging); Mist-style anomaly
insights (client-count/CPU vs same-hour 14-day baseline + error bursts) in the
Operations engine.
Firmware orchestration: staged wave-based RouterOS rollouts (canary first) with
pre-upgrade backup, reboot-ride-out, version verification, halt-on-failure, and
scheduling; live per-device progress + cancel.
Platform & automation: scoped API tokens (mtm_, read/write, hashed at rest,
Bearer auth in requireAuth); outbound webhooks (12 events, HMAC-signed) fired
from the alert pipeline + firmware orchestrator; scheduled email reports.
api-ssl: RouterOSClient negotiates TLS on port 8729 and accepts RouterOS's
default self-signed certs (rejectUnauthorized:false), so SSL-secured API devices
connect without manual cert trust. Fixes#57.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Configuration History:
- Per-device snapshots from the full RouterOS /export (.rsc) — config-only, so
counters/operational state no longer cause phantom changes; deduped by content hash
- New Config History tab: snapshot timeline, readable line diff, on-demand Capture
(with honest 'no changes' feedback), one-click rollback via the linked .rsc backup
- Snapshot and its backup are one artifact: badged 'Config Snapshot' on the Backups
page, FK ON DELETE CASCADE keeps them in lockstep; delete is gated by a warning modal
- New config_drift alert event (off by default); periodic snapshot poll with
configurable cadence/retention (config_snapshot_interval_min/_retention)
- New endpoints under /api/config-history (list/get/diff/capture/rollback/delete)
Scheduled Backups:
- Replaced the raw cron textbox in Settings with Frequency (Daily/Weekly/Monthly)
+ time + day pickers; cron is generated/parsed behind the scenes
- Fixed cronMatchesNow to evaluate day-of-month/month/day-of-week (it previously
only checked minute/hour, so weekly/monthly schedules ran every day)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(security): default-secret banners, login rate limiting, real-IP passthrough
- app.set('trust proxy', 1) so req.ip resolves to real client IP behind nginx
- loginRateLimit() middleware on POST /api/auth/login; limits read from
app_settings (login_rate_limit_max / login_rate_limit_window_sec) with
60-second cache; falls back to 10/60s when Redis/DB unavailable
- GET /api/auth/security-status returns warnings for default JWT_SECRET,
ENCRYPTION_KEY, and admin/admin credential — non-breaking for dev
- SecurityBanners component in AppLayout: yellow banner for default secrets,
red banner + change-password link for default admin password
- Login rate limit settings exposed in Settings > General tab
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): rate-limit /security-status, suppress CodeQL false positive on /login
- Add allMethods option to rateLimitRedis so it can apply to GET routes
- Apply rateLimitRedis (10 req/60s per user) to GET /auth/security-status
to prevent bcrypt-based CPU exhaustion by authenticated users
- Add lgtm suppression comment on POST /login — CodeQL does not recognize
our custom loginRateLimit() middleware; rate limiting is in place
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Denis Evers <git@evers.sh>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Squash-merge of PR #28 from @denis-ev. All three review blockers addressed: encryption key management documented, string truncation replaced with skip+warn validation, and undefined return on duplicate merge fixed with proper 500. Local test stack verified against cloned prod DB — migrations, new features, and backend all clean.
2GT-RichandClaude Opus 5
