From e10f7ba56804a56cc7a0228383638bd9958cb4a3 Mon Sep 17 00:00:00 2001 From: 2GT-Rich <107219997+2GT-Rich@users.noreply.github.com> Date: Fri, 21 Aug 2026 12:26:18 -0700 Subject: [PATCH] docs: split reference material into docs/, keeping the README a shop window (v0.23.4-beta) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documentation only — no functional change. The README was carrying two jobs for two different readers. Its Reference section was the single largest block at 154 lines, and nobody deciding whether to try the project reads an InfluxDB environment variable table — while the people who need it were scrolling past the pitch to reach it. Split along that seam: anything you would read before installing stays in the README, anything you would read after installing moves to docs/. README.md 430 lines sell, prove, install, feature overview docs/change-guard.md 193 safety system, settings, guard states docs/architecture.md 116 data stores, polling model, module layout docs/capsman.md 84 role model, radio matching, current limits docs/alerting.md 81 rules, channels, webhooks, maintenance windows docs/api.md 77 scoped tokens, preflight, scripting docs/configuration.md 76 environment, secrets, rotation, TLS docs/sso-oidc.md 71 identity provider setup and behaviour docs/README.md 20 index In-repo rather than a documentation site, deliberately. A change to Change Guard's semantics can edit its documentation in the same commit as the code — which, given how often that behaviour has been refined recently, is the property that actually matters. A static site generator can point at docs/ later without moving a file. The Change Guard pitch stays on the front page: the problem framing, the verified recovery timeline and the quoted prediction warning are what make someone care, and a reference page is the wrong home for them. Three sections are new rather than relocated, written because the split exposed the gaps: - architecture.md did not exist. It now records why there are three data stores, the polling cadences, and which modules are pure functions over a snapshot and why. - Scoped API tokens shipped without usage documentation. api.md shows preflighting a change from a script and reading the guard block afterwards. - change-guard.md records why ip-on-bridge-port is informational rather than critical on RouterOS 7 — reasoning that previously existed only in a commit message. Every internal anchor and cross-file link verified to resolve, including anchors into other documents. No content dropped, checked by keyword comparison against the previous single file. Co-Authored-By: Claude Opus 5 --- README.md | 185 +++++--------------------------------- backend/package.json | 2 +- docs/README.md | 20 +++++ docs/alerting.md | 81 +++++++++++++++++ docs/api.md | 77 ++++++++++++++++ docs/architecture.md | 116 ++++++++++++++++++++++++ docs/capsman.md | 84 +++++++++++++++++ docs/change-guard.md | 193 ++++++++++++++++++++++++++++++++++++++++ docs/configuration.md | 76 ++++++++++++++++ docs/sso-oidc.md | 71 +++++++++++++++ frontend/package.json | 2 +- frontend/src/version.ts | 2 +- 12 files changed, 742 insertions(+), 167 deletions(-) create mode 100644 docs/README.md create mode 100644 docs/alerting.md create mode 100644 docs/api.md create mode 100644 docs/architecture.md create mode 100644 docs/capsman.md create mode 100644 docs/change-guard.md create mode 100644 docs/configuration.md create mode 100644 docs/sso-oidc.md diff --git a/README.md b/README.md index 57357e3..6448570 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@

- Version + Version License Docker TypeScript @@ -33,7 +33,7 @@ - [Visibility and troubleshooting](#visibility-and-troubleshooting) - [Fleet operations](#fleet-operations) - [Access control and platform](#access-control-and-platform) -- [Reference](#reference) +- [Documentation](#documentation) - [Contributing](#contributing) - [License](#license) @@ -259,28 +259,9 @@ The layers described [above](#what-makes-it-different), plus the surrounding mac - **Capability probe** — `POST /api/devices/:id/change-guard/probe` reports what safety mechanisms a device actually supports, tested rather than assumed, and leaves nothing behind - **Guard history** — every protected change recorded per device as `committed`, `reverted`, or `failed` -

-What you see when auto-revert fires - -
- -The request returns **HTTP 200** with `guard.auto_reverting: true` and *"Contact with the device was -lost while applying this change. It is restoring itself and should come back shortly."* — not an error. - -This matters: the most dangerous changes sever the very connection carrying them, so the API call times -out even though the change succeeded on the device. **Reachability, not the exception, decides the -outcome.** A change that throws while the device is still reachable is a genuine failure, and the guard -is disarmed immediately so nothing reboots for nothing. - -Two consequences worth knowing before relying on it: - -- **Binary mode reboots the device** to restore. That is the price of a restore that recovers deleted - interfaces exactly. Script mode (`/export` + `/import`) avoids the reboot but is additive, so it - cannot recover a deletion. -- **The restore point is saved before the scheduler is armed**, so a rollback returns a configuration - containing neither the scheduler nor the change. Self-cleaning, and a revert loop is impossible. - -
+For the full behaviour — what the API returns when contact is lost, how binary and script +modes differ, and how to choose a timeout — see +[Change Guard and Config Health](docs/change-guard.md). ### Monitoring and operations @@ -306,7 +287,7 @@ Two consequences worth knowing before relying on it: - **Spectral and AP scans** — scheduled or on demand, per radio - **Rogue & neighbour AP detection** — stored scan results cross-referenced against your own SSIDs and radio MACs; a foreign BSSID broadcasting your SSID is flagged as an evil twin - **Device fingerprinting** — every client classified into a category (server, computer, phone, TV, camera, printer, console, IoT…) from OUI and hostname, overridable per client and persistent across polls -- **CAPsMAN** — role detection (standalone / CAP / controller / controller-with-local-radios), provisioned interfaces labelled with their controller, and a controller panel grouping every AP with its radios, live channel, provisioned SSID and client counts. CAPs are matched to managed devices by radio MAC — hardware identity, so it cannot confuse two devices on two segments. Read-only for now; see [CAPsMAN scope](#capsman-scope) +- **CAPsMAN** — role detection (standalone / CAP / controller / controller-with-local-radios), provisioned interfaces labelled with their controller, and a controller panel grouping every AP with its radios, live channel, provisioned SSID and client counts. CAPs are matched to managed devices by radio MAC — hardware identity, so it cannot confuse two devices on two segments. Read-only for now; see [CAPsMAN scope](docs/capsman.md#current-limits) - **Guest WiFi** — a guided wizard that builds a full captive portal in one pass: guest SSID on every radio, VLAN segregation, IP pool, DHCP, hotspot profile, bandwidth-limited user profile and optional NAT. Plus batch vouchers with printable 3-up sheets, a live guests-online table, and an inline walled garden ### Firewall and network services @@ -367,120 +348,30 @@ Each network service supports multi-device management with conflict detection: - **Roles** — Admin, Operator (read/write), Viewer (read-only) - **Two-factor authentication** — per-user TOTP via QR code, required at login once enabled -- **Single sign-on (OIDC)** — any standards-compliant provider (Entra ID, Okta, Google Workspace, Keycloak, Authentik…), with group-to-role mapping, auto-provisioning, and local login retained as break-glass. Configured entirely in the UI — see [Single sign-on setup](#single-sign-on-oidc) +- **Single sign-on (OIDC)** — any standards-compliant provider (Entra ID, Okta, Google Workspace, Keycloak, Authentik…), with group-to-role mapping, auto-provisioning, and local login retained as break-glass. Configured entirely in the UI — see [the SSO guide](docs/sso-oidc.md) - **Scoped API tokens** — `mtm_…` tokens with read or write scope and optional expiry, shown once and stored only as a SHA-256 hash. No token can perform admin actions or manage other tokens - **Outbound webhooks** — subscribe to any of twelve events, delivered as HMAC-SHA256-signed JSON POSTs through the same pipeline that respects alert rules, cooldowns and maintenance windows - **Scheduled email reports** — daily, weekly or monthly HTML fleet summaries to any recipient list - **Credential presets** — shared device credentials, optionally restricted to admins only -- **Encryption at rest** — device passwords encrypted with AES-256-GCM under a [self-healing key](#secret-management-self-healing) +- **Encryption at rest** — device passwords encrypted with AES-256-GCM under a [self-healing key](docs/configuration.md#secret-management-self-healing) - **TLS** — self-signed certificate generated on first run, replaceable via the Settings UI; nginx terminates TLS and redirects HTTP --- -## Reference +## Documentation -### Environment variables +Deeper reference material lives in [`docs/`](docs/README.md), versioned alongside the code +so it moves when behaviour does. -Set in `.env`. Both secrets auto-generate on first boot if you leave them unset. - -| Variable | Default | Description | -|---|---|---| -| `JWT_SECRET` | *auto-generated & persisted* | Signs session tokens. Set it to pin your own value. | -| `ENCRYPTION_KEY` | *auto-generated & persisted* | Encrypts device passwords at rest. Set it to pin your own value. | -| `CORS_ORIGIN` | *localhost defaults* | Comma-separated browser origins allowed to call the API. **Required in production.** | -| `DB_PASSWORD` | `mikrotik_secure_pw` | PostgreSQL password | -| `INFLUXDB_TOKEN` | `mytoken123456789` | InfluxDB admin token | -| `INFLUXDB_ORG` | `mikrotik-manager` | InfluxDB organization | -| `INFLUXDB_BUCKET` | `metrics` | InfluxDB bucket for time-series data | -| `INFLUXDB_ADMIN_PASSWORD` | `admin_password_123` | InfluxDB admin UI password | -| `HTTP_PORT` | `80` | Host port for HTTP (redirects to HTTPS) | -| `HTTPS_PORT` | `443` | Host port for HTTPS | - -> Never commit `.env`. It is already in `.gitignore`. - -### Change Guard and Config Health settings - -These live in the Settings UI rather than `.env`, so they apply without a restart. - -| Setting | Default | Description | -|---|---|---| -| `change_guard_enabled` | `true` | Master switch. When off, guarded changes still apply — with no safety net and no lockout refusal. | -| `change_guard_mode` | `binary` | `binary` = `/system backup` restore; recovers deletions exactly, **reboots the device**. `script` = `/export` + `/import`; no reboot, but additive, so it cannot undo a deletion. | -| `change_guard_timeout_sec` | `120` | How long the device waits before rescuing itself. Must exceed the manager's verification window (~40s) with margin for a slow device. | -| `config_health_enabled` | `true` | Runs the standing configuration audit. | -| `config_health_interval_min` | `60` | Audit cadence per device — one read-only snapshot over the API. | - -Overriding a predicted lockout is deliberate by design: the API requires `confirm_lockout: true` and the -UI requires typing the device name. The change then runs under Change Guard regardless, so an override -is a decision to *rely on* auto-revert — not to bypass protection. - -### CAPsMAN scope - -Provisioning, configuration edits and interface enable/disable are **deliberately not exposed**. A -CAPsMAN configuration applies to every access point bound to it simultaneously, which is the fleet-wide -version of the failure Change Guard exists to prevent. Those operations are held back until they carry -the same prediction-and-revert protection that per-device changes do. - -Legacy `/caps-man` (the pre-RouterOS 7 controller) is a separate API tree and is not covered yet. - -### Secret management (self-healing) - -`JWT_SECRET` and `ENCRYPTION_KEY` are managed automatically: - -- **Set to a strong value in the environment** — that value is used, and you stay in control. -- **Unset or left at an old default** — the backend generates a strong secret once, persists it to the - `app_data` volume (`SECRETS_DIR`, default `/app/data`), and reuses it every boot. Secrets live outside - the database, so a database dump alone cannot reveal the key protecting the credentials inside it. - -Upgrades are non-breaking. Existing ciphertext is decrypted via a **legacy-key fallback** (including -previous defaults) and transparently re-encrypted under the current key by a background sweep. Rotating -the JWT secret off a public default invalidates sessions signed with it — users simply log in again. - -**Key rotation:** set a new `ENCRYPTION_KEY` (or delete the persisted secret to force regeneration) and -restart. Old rows keep decrypting via the fallback and are re-encrypted forward automatically. **If the -persisted secret and every prior key are lost**, ciphertext under that key cannot be recovered — re-enter -device credentials or restore a backup. - -### Single sign-on (OIDC) - -Authorization Code flow with PKCE. The server validates the ID token (JWKS signature plus `iss`, `aud`, -`exp`, `nonce`) and then issues the platform's normal session, so SSO users get the same experience as -local ones. **No environment variables or redeploy** — everything is configured in the UI, with the -client secret encrypted at rest. - -**1. Register the app with your provider.** Create an OIDC web application and note its **Issuer URL** -(the base serving `/.well-known/openid-configuration`), **Client ID** and **Client Secret** — public -clients with no secret work too, via PKCE. Use this redirect URI: - -``` -https:///api/auth/oidc/callback -``` - -**2. Configure it under Settings → SSO / OIDC.** - -| Field | Purpose | +| Document | Covers | |---|---| -| **Issuer URL** | Your provider's base URL; **Test discovery** verifies reachability. | -| **Client ID / Secret** | From step 1. The secret is write-only; leave blank to keep the existing one. | -| **Scopes** | Default `openid profile email`. Add a groups scope if your IdP needs one. | -| **Username / Email / Groups claim** | Defaults `preferred_username`, `email`, `groups`. | -| **Group → role mapping** | Map IdP groups to admin / operator / viewer. Highest-privilege match wins. | -| **Default role** | For users matching no mapping (default **viewer**). | -| **Auto-provision** | Create a local user on first sign-in. | -| **Link by verified email** | Attach an SSO identity to an existing account when `email_verified` matches. | -| **Allowed email domains** | Optional allowlist. | -| **Button label** | Text on the login button, e.g. "Sign in with Okta". | - -**Behaviour and safety:** - -- **Provisioning** — first sign-in creates a user, or links to an existing local account when the - verified email matches. SSO users have no local password. -- **Roles** are always resolved server-side from your mapping, and existing users are never silently - demoted when a login carries no matching group. -- **Break-glass** — local login stays available, so the seeded admin can always get in even if the IdP - is misconfigured or unreachable. -- **Behind a proxy** — the redirect URI derives from the request host; set **Public base URL** to pin it - when your external URL differs from what the backend sees. +| [Configuration](docs/configuration.md) | Environment variables, secret management, key rotation, TLS | +| [Change Guard and Config Health](docs/change-guard.md) | How the safety system works, its settings, and what to expect when it fires | +| [Alerting](docs/alerting.md) | Rules, channels, ntfy priority mapping, webhooks, maintenance windows | +| [Single sign-on (OIDC)](docs/sso-oidc.md) | Identity provider setup, group-to-role mapping, break-glass behaviour | +| [CAPsMAN](docs/capsman.md) | How centrally provisioned APs are modelled, and current limits | +| [API and automation](docs/api.md) | Scoped tokens, preflighting a change, driving the platform from scripts | +| [Architecture](docs/architecture.md) | Data stores, polling model, and where the interesting logic lives | ### Tech stack @@ -489,45 +380,11 @@ https:///api/auth/oidc/callback | **Frontend** | React 18, TypeScript, Vite, Tailwind CSS | | **State / data** | TanStack Query v5, React Router v6, Zustand | | **Charts / topology / maps** | Recharts, @xyflow/react, Leaflet | -| **Terminal** | xterm.js | | **Backend** | Node.js, Express, TypeScript | -| **Primary DB** | PostgreSQL 15 | -| **Time-series DB** | InfluxDB 2.7 | -| **Cache / queue** | Redis 7, BullMQ | +| **Data stores** | PostgreSQL 15, InfluxDB 2.7, Redis 7 + BullMQ | | **Real-time** | Socket.IO | | **Device comms** | RouterOS API (8728 / 8729), SSH2 | -| **Proxy** | nginx — TLS termination and static serving | -| **Container** | Docker Compose | - -### Project structure - -``` -mikrotik-manager/ -├── frontend/ # React + TypeScript (Vite) -│ └── src/ -│ ├── pages/ # One component per route -│ ├── components/ # Shared UI -│ ├── services/ # API client (Axios) -│ ├── hooks/ # Custom React hooks -│ └── types/ # TypeScript definitions -│ -├── backend/ # Node.js + Express + TypeScript -│ └── src/ -│ ├── routes/ # REST API handlers -│ ├── services/ # Polling, alerts, backups, orchestration -│ │ ├── mikrotik/ # RouterOS API client and device collector -│ │ ├── changeGuard/# Auto-revert, lockout prediction, Config Health -│ │ └── topology/ # Neighbour graph construction -│ ├── db/ # Schema and migrations -│ ├── config/ # Database, InfluxDB, Redis connections -│ ├── middleware/ # Auth, audit logging, error handling -│ └── utils/ # Crypto, OUI lookup, VLAN parsing -│ -├── nginx/ # Reverse proxy config and Dockerfile -├── docker-compose.yml # Build from source -├── docker-compose.ghcr.yml # Pre-built images -└── .env.example -``` +| **Proxy / container** | nginx, Docker Compose | --- diff --git a/backend/package.json b/backend/package.json index 07d23be..2fbc3e3 100644 --- a/backend/package.json +++ b/backend/package.json @@ -1,6 +1,6 @@ { "name": "mikrotik-manager-backend", - "version": "0.23.3-beta", + "version": "0.23.4-beta", "description": "Mikrotik Network Management Platform - Backend API", "main": "dist/index.js", "scripts": { diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..fcf68a3 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,20 @@ +# MikroTik Manager documentation + +Reference material for people already running the platform. If you are still deciding +whether to try it, the [project README](../README.md) is the better starting point. + +| Document | Covers | +|---|---| +| [Configuration](configuration.md) | Environment variables, secret management and key rotation | +| [Change Guard and Config Health](change-guard.md) | How the safety system works, its settings, and what to expect when it fires | +| [Alerting](alerting.md) | Alert rules, delivery channels, webhooks, scheduled reports, maintenance windows | +| [Single sign-on (OIDC)](sso-oidc.md) | Identity provider setup, group-to-role mapping, break-glass behaviour | +| [CAPsMAN](capsman.md) | How centrally provisioned access points are modelled, and current limits | +| [API and automation](api.md) | Scoped tokens, authentication, and driving the platform from scripts | +| [Architecture](architecture.md) | How the system is put together, for contributors | + +## Conventions used here + +- **Settings in the UI** are stored in the database and apply without a restart. +- **Environment variables** live in `.env` and require a container restart to change. +- Anything described as *read-only* makes no writes to your devices. diff --git a/docs/alerting.md b/docs/alerting.md new file mode 100644 index 0000000..520f36a --- /dev/null +++ b/docs/alerting.md @@ -0,0 +1,81 @@ +# Alerting + +[← Documentation index](README.md) + +## Alert rules + +Each rule can be enabled independently, with its own threshold where applicable and a +cooldown that prevents a flapping device from flooding your channels. + +| Event | Notes | +|---|---| +| `device_offline` / `device_online` | Raised by the poller; recovery closes the outage record | +| `high_cpu` / `high_memory` | Configurable threshold | +| `cert_expiry` | Certificate approaching expiry | +| `firmware_update_available` | A newer RouterOS release exists for the device | +| `log_error` / `log_warning` | Matched from the device's own log | +| `device_discovered` | An unmanaged neighbour appeared via LLDP/CDP/MNDP | +| `config_drift` | The device's configuration changed (off by default) | + +Alerts are suppressed for devices inside an active [maintenance window](#maintenance-windows). + +## Delivery channels + +Email, Slack, Discord, Telegram and **ntfy**. Channels are configured under +**Settings → Alerts → Channels**; secrets are masked on read and preserved when you save +a channel without retyping them. + +### ntfy + +Works with the public `ntfy.sh` or your own instance. + +| Field | Notes | +|---|---| +| **Server URL** | Blank for `https://ntfy.sh`, or your own instance | +| **Topic** | Required | +| **Access token** | `tk_…`; preferred, since it can be scoped and revoked | +| **Username / password** | Basic-auth alternative | +| **Manager URL** | Optional — makes each notification tappable, opening the device it refers to | + +Authentication is optional, but an unprotected topic means anyone who guesses the name +can publish to it, so use one of the two. + +**Priority mapping.** Severity maps onto ntfy's 1–5 scale so the channel stays worth +being woken by: + +| Events | Priority | +|---|---| +| Device offline, log errors, CPU/memory pressure | **4** — breaks through do-not-disturb | +| Certificate expiry, config drift, firmware available | 3 — default | +| Device recovery, new device discovered | 2 — low | + +Each notification carries its event type as a **tag**, so a client can be filtered to +wake only for the events you care about rather than needing a channel per event. + +## Outbound webhooks + +Subscribe any URL to twelve events: device up/down, log errors, high CPU, high memory, +certificate expiry, device discovered, firmware update available, config drift, and +firmware rollout completed/failed. + +Deliveries are JSON `POST`s, **HMAC-SHA256 signed** in `X-MTM-Signature` when a secret is +set. Last-delivery status is tracked per webhook and there is a Send-test button. + +Webhooks fire through the same pipeline as other alerts, so they respect alert rules, +cooldowns and maintenance windows rather than bypassing them. + +## Scheduled email reports + +Daily, weekly or monthly HTML fleet summaries to any recipient list, using the same SMTP +settings as email alerts. Each report covers devices online, outages and total downtime, +error and warning counts, updates pending, backups taken, and top clients by traffic. +Send-now is available for an immediate copy. + +## Maintenance windows + +Schedule planned downtime per device, or across a group, so alerts are suppressed +automatically rather than being muted globally and forgotten. + +- One-time or recurring (cron-based) +- Active windows can be deactivated early +- Managed under **Settings → Maintenance** diff --git a/docs/api.md b/docs/api.md new file mode 100644 index 0000000..e546b32 --- /dev/null +++ b/docs/api.md @@ -0,0 +1,77 @@ +# API and automation + +[← Documentation index](README.md) + +The web UI is a client of the same REST API you can drive yourself. Anything the interface +does is available to a script. + +## Scoped API tokens + +Issue tokens under **Settings → Automation**. + +- Prefix `mtm_…`, with **read** or **write** scope and an optional expiry +- Shown **once** on creation — only a SHA-256 hash is stored +- Mapped onto the role model: no token can perform admin actions or manage other tokens, + regardless of scope + +```bash +curl -sk https://manager.example.com/api/devices \ + -H "Authorization: Bearer mtm_your_token_here" +``` + +A token with `read` scope is rejected on any mutating request, so a monitoring integration +cannot change your network even if the token leaks. + +## Session authentication + +The UI authenticates with a JWT obtained from `/api/auth/login`, optionally followed by +`/api/auth/totp/verify` when two-factor is enabled. Tokens are bearer credentials and are +sent in the same `Authorization` header. + +For scripting, prefer an API token — it is scoped, revocable, and unaffected by password +or 2FA changes. + +## Endpoints worth knowing + +| Endpoint | Purpose | +|---|---| +| `GET /api/devices` | Fleet inventory with status, model, versions and serials | +| `POST /api/devices/:id/sync` | Force an immediate full collection for one device | +| `GET /api/devices/:id/management-path` | How the manager reaches a device, with the reason for each hop | +| `POST /api/devices/:id/preflight` | Analyse a change **without applying it** | +| `POST /api/devices/:id/change-guard/probe` | Report which safety mechanisms the device supports | +| `GET /api/devices/:id/config-health` | Latest standing-audit findings | +| `GET /api/topology` | Graph of devices, links, external nodes, and distrusted identifiers | +| `GET /api/operations/insights` | The dashboard's "things to handle" feed | + +`preflight` is the useful one for automation: it returns the same verdict the UI shows, +so a pipeline can refuse its own change before touching the device. See +[Change Guard](change-guard.md#lockout-prediction). + +## Applying a guarded change from a script + +Guarded endpoints return **409** with `lockout: true` and a verdict when a change is +predicted to sever management. To proceed anyway, resend with `confirm_lockout: true` — +the change then runs under Change Guard, so a mistake still recovers itself. + +```bash +# Refused, with an explanation +curl -sk -X PUT https://manager.example.com/api/devices/8/ports/ether1/vlan \ + -H "Authorization: Bearer mtm_…" -H 'Content-Type: application/json' \ + -d '{"pvid":99,"tagged_vlans":[],"untagged_vlans":[99]}' + +# Accepted, relying on auto-revert +curl -sk -X PUT https://manager.example.com/api/devices/8/ports/ether1/vlan \ + -H "Authorization: Bearer mtm_…" -H 'Content-Type: application/json' \ + -d '{"pvid":99,"tagged_vlans":[],"untagged_vlans":[99],"confirm_lockout":true}' +``` + +A successful guarded change returns a `guard` block describing what happened — +`confirmed`, `auto_reverting`, or an `unprotected_reason` when the safety net could not be +armed. Check it rather than relying on the HTTP status alone. + +## Webhooks + +For push rather than poll, see [Alerting → outbound webhooks](alerting.md#outbound-webhooks). +Deliveries are HMAC-SHA256 signed and respect alert rules, cooldowns and maintenance +windows. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..8c33c2d --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,116 @@ +# Architecture + +[← Documentation index](README.md) + +For contributors, and for anyone wanting to know what happens between the UI and a device. + +## The shape of it + +``` +Browser ──► nginx ──► Express API ──► RouterOS API / SSH ──► devices + │ + ├──► PostgreSQL current state, config, users, audit + ├──► InfluxDB time-series metrics + └──► Redis job queue and locks +``` + +Three data stores, because they answer different questions: + +| Store | Holds | Why not the others | +|---|---|---| +| **PostgreSQL** | Devices, interfaces, VLANs, clients, users, audit log, findings | Relational, needs joins and constraints | +| **InfluxDB** | CPU, memory, traffic, client counts, RF quality | Time-series at poll cadence; would bloat Postgres | +| **Redis** | BullMQ job queues, per-device locks, poll gating | Ephemeral, needs atomic operations | + +## Polling + +A scheduler enqueues work per device on separate cadences, so a slow operation never +delays a fast one: + +| Cycle | Default | Collects | +|---|---|---| +| Fast | 30s | Reachability, CPU, memory, traffic counters, clients | +| Slow | 5min | Interfaces, VLANs, addresses, neighbours, STP, wireless, CAPsMAN | +| Logs | 60s | Device log entries, feeding log alerts | +| Config snapshot | 60min | `/export`, deduplicated by content hash | +| Config health | 60min | Read-only configuration audit | +| MAC / spectral / AP scan | configurable | Switch MAC tables, RF surveys | + +Devices are probed once for wireless capability and the result recorded, so non-wireless +devices are skipped cheaply on every later poll. + +## Device communication + +`RouterOSClient` speaks the RouterOS **binary API** (8728, or 8729 with TLS). It is used +for essentially everything, including the safety machinery, because many deployments never +configure SSH. + +SSH is used only where RouterOS offers no API equivalent — `/export` for config snapshots, +binary backup retrieval, packet capture download, and the in-browser terminal. + +## Where the interesting logic lives + +``` +backend/src/services/ +├── mikrotik/ +│ ├── RouterOSClient.ts binary protocol client +│ ├── DeviceCollector.ts all collection and device mutation +│ └── capsman.ts role detection, radio matching (pure) +├── changeGuard/ +│ ├── ChangeGuard.ts restore point, scheduler, verify, commit +│ ├── pathModel.ts live snapshot, management-path resolution (pure) +│ ├── invariants.ts what must hold for management to survive (pure) +│ ├── analyzeChange.ts simulate a change, diff the invariants (pure) +│ └── configHealth.ts standing audit rules (pure) +├── topology/ +│ └── buildTopology.ts neighbour graph construction (pure) +├── netflow/ NetFlow v9 / IPFIX collector and aggregator +└── PollerService.ts scheduling and workers +``` + +Anything marked *pure* is a function over a snapshot with no I/O. That is deliberate: this +logic is subtle, its failure mode is a plausible-looking wrong answer rather than a crash, +and it must be testable without the hardware it reasons about. Those modules carry the bulk +of the test suite. + +## Repository layout + +``` +mikrotik-manager/ +├── frontend/ # React + TypeScript (Vite) +│ └── src/ +│ ├── pages/ # One component per route +│ ├── components/ # Shared UI +│ ├── services/ # API client (Axios) +│ ├── hooks/ # Custom React hooks +│ └── types/ # TypeScript definitions +│ +├── backend/ # Node.js + Express + TypeScript +│ └── src/ +│ ├── routes/ # REST API handlers +│ ├── services/ # See above +│ ├── db/ # Schema and migrations +│ ├── config/ # Database, InfluxDB, Redis connections +│ ├── middleware/ # Auth, audit logging, error handling +│ └── utils/ # Crypto, OUI lookup, VLAN parsing +│ +├── docs/ # This documentation +├── nginx/ # Reverse proxy config and Dockerfile +├── scripts/ # CI preflight and tooling +├── docker-compose.yml # Build from source +├── docker-compose.ghcr.yml # Pre-built images +└── .env.example +``` + +## Database migrations + +A single idempotent SQL script in `backend/src/db/migrate.ts` runs on every backend start. +Statements are written to be safe to re-run — `CREATE TABLE IF NOT EXISTS`, +`ADD COLUMN IF NOT EXISTS`, and guarded constraint swaps — so upgrading is just starting +the new image. + +## Before you push + +`scripts/ci-preflight.sh` runs the same gates as CI — lint, type-check, build, tests and +production audits for both packages — in about 35 seconds. Running it locally is faster +than finding out from a red badge. diff --git a/docs/capsman.md b/docs/capsman.md new file mode 100644 index 0000000..fee0598 --- /dev/null +++ b/docs/capsman.md @@ -0,0 +1,84 @@ +# CAPsMAN + +[← Documentation index](README.md) + +An access point provisioned by CAPsMAN holds none of its own wireless configuration — +the controller owns it. Reading the AP directly returns blank SSID, security and band, +which makes a perfectly healthy access point look broken. + +The platform models the controller relationship instead. + +## Device roles + +Every device is classified each poll from `/interface/wifi/capsman` and +`/interface/wifi/cap`: + +| Role | Meaning | +|---|---| +| `standalone` | Has radios, manages them itself | +| `cap` | Radios are provisioned by a controller | +| `controller` | Runs CAPsMAN; has no radios of its own | +| `controller_cap` | Runs CAPsMAN **and** its own radios under it — a supported MikroTik arrangement | +| `none` | No wireless hardware | + +The role is detected, not inferred from `device_type`. That matters because a dedicated +controller is usually classified as a router, and `device_type` is a label a human typed +rather than a fact about the hardware. The Radios tab and the wireless section follow the +detected role. + +## How CAPs are matched to devices + +The controller enumerates every radio it manages — local and remote — through +`/interface/wifi/radio`, each with a `radio-mac`. Those MACs are matched against every +interface MAC in the fleet. + +MAC rather than IP address, deliberately: a MAC is hardware identity and unique +fleet-wide, whereas an address is only unique within a broadcast domain. An installation +with several segments reusing the same addressing would otherwise attribute a radio to +the wrong device entirely. + +Two refinements come from real deployments: + +- A controller **mirrors each CAP's interfaces locally**, so a CAP's radio MAC genuinely + appears on two devices. A radio the controller reports as `local: false` lives on a CAP + by definition, so the controller is never a valid answer for it. +- A radio MAC often differs from the device's interface MAC in the final octet, so lookup + falls back to a five-octet prefix while keeping the OUI exact. + +A CAP that is **not** in the fleet is surfaced as unmanaged rather than hidden — that is +usually the reason an access point appears to be missing. + +## What you get + +The **CAPsMAN panel** on the Wireless page lists every controller with the access points +it provisions, one row per radio: the AP it lives on (linked), interface name, provisioned +SSID, operating channel, connected clients, and state. + +Two details are read from the controller rather than the CAP, because the CAP does not +know them: + +- **SSID** — held on the controller's mirror of the interface. +- **Client counts** — a CAP's own registration table is empty when traffic is processed + centrally. Counts come from the registration table with each client attributed to its + radio by following `master-interface`, since clients register on the virtual AP carrying + the SSID rather than on the physical radio. + +## Write protection + +A provisioned radio is owned by the controller, so a local write to it is either rejected +outright or silently replaced at the next provision — while the API reports success. + +Five paths refuse or skip provisioned radios, naming the controller in the error: bulk +SSID deployment, interface create, interface edit, interface delete, and guest network +setup. + +## Current limits + +**Provisioning, configuration edits and interface enable/disable are deliberately not +exposed.** A CAPsMAN configuration applies to every access point bound to it +simultaneously — the fleet-wide version of the failure [Change Guard](change-guard.md) +exists to prevent. Those operations are held back until they carry the same +prediction-and-revert protection that per-device changes do. + +Legacy `/caps-man` — the pre-RouterOS 7 controller — is a separate API tree and is not +covered. The RouterOS 7 `wifi` stack is supported. diff --git a/docs/change-guard.md b/docs/change-guard.md new file mode 100644 index 0000000..e07c3e6 --- /dev/null +++ b/docs/change-guard.md @@ -0,0 +1,193 @@ +# Change Guard and Config Health + +[← Documentation index](README.md) + +RouterOS applies every command immediately and independently. There is no transaction, +no rollback, and no cross-object validation — so it will accept a change that severs the +very path you are managing the device over. MikroTik's own documented advice for enabling +VLAN filtering is to have a serial console ready. + +Three layers address that. This page covers how they behave and how to configure them; +the [README](../README.md#what-makes-it-different) has the short version. + +## Change Guard — device-side auto-revert + +Before applying a protected change: + +1. The device saves a **restore point** locally. +2. A **scheduler** is armed to reapply it after a timeout. +3. The change is applied. +4. Reachability is proven **on a brand-new connection** — not the one that made the change. +5. Confirmed → the scheduler is disarmed and the restore point deleted. + Unreachable → nothing is done, and the device restores itself. + +The restore point is saved *before* the scheduler is armed, so a rollback returns a +configuration containing neither the scheduler nor the change. It is self-cleaning, and +a revert loop is impossible. + +Everything runs over the RouterOS API, so it works on devices with no SSH credentials +configured. + +### Covered changes + +Twelve change types are guarded: + +| Area | Operations | +|---|---| +| Bridge | VLAN filtering toggle | +| Ports | PVID and tagged/untagged membership | +| Bridge VLANs | add, update, delete | +| Addressing | IP address add, remove | +| Routing | route add, remove | +| Bonding | bond create, delete | +| Services | management service enable/disable | + +### What you see when it fires + +Verified end to end on a CRS running RouterOS 7.23.3 by deliberately deleting a switch's +own management address: + +``` +T+0 restore point saved, revert scheduler armed, change applied +T+0s device drops off the network +T+40s manager gives up after 4 fresh connection attempts and reports back +T+120s device restores its own backup and reboots +T+3m device is back, change undone, scheduler and restore point gone +``` + +The request returns **HTTP 200** with `guard.auto_reverting: true` and the message +*"Contact with the device was lost while applying this change. It is restoring itself and +should come back shortly."* — deliberately not an error. + +This matters. The most dangerous changes sever the very connection carrying them, so the +API call times out even though the change succeeded on the device. **Reachability, not the +exception, decides the outcome.** A change that throws while the device is still reachable +is a genuine failure, and the guard is disarmed immediately so nothing reboots for nothing. + +### Guard history + +Every protected change is recorded per device: + +| Status | Meaning | +|---|---| +| `committed` | Change applied, device confirmed reachable, guard disarmed | +| `reverted` | Contact lost; the device was left to restore itself | +| `failed` | Change rejected by the device; nothing applied, guard disarmed | + +### Settings + +| Setting | Default | Description | +|---|---|---| +| `change_guard_enabled` | `true` | Master switch. When off, guarded changes still apply — with no safety net and no lockout refusal. | +| `change_guard_mode` | `binary` | See below. | +| `change_guard_timeout_sec` | `120` | How long the device waits before rescuing itself. | + +**Choosing a mode:** + +| Mode | Mechanism | Reboots? | Recovers a deletion? | +|---|---|---|---| +| `binary` | `/system backup` restore | **Yes** | Yes — restores the configuration exactly | +| `script` | `/export` then `/import` | No | **No** — `/import` is additive | + +`binary` is the default because a change that deletes an interface can only be undone by +a restore that recreates it. `script` avoids the reboot but cannot undo a deletion, which +is the case most likely to lock you out. + +**Choosing a timeout:** it must exceed the time the manager needs to verify reachability — +roughly 40 seconds across four connection attempts — with margin for a slow device. Too +short and a healthy device reverts a good change; too long and an outage lasts longer than +it needs to. + +## Lockout prediction + +Rather than blocklisting operations someone once decided were dangerous, the platform +reads live device state, resolves how the manager actually reaches the device, simulates +the proposed change against that model, and reports any invariant that flips from +satisfied to violated. + +Two details make the result precise rather than merely cautious: + +- The **manager's own address**, as the device sees it, is read from the device's + connection tracking — rather than being treated as unknowable behind NAT. +- The **ingress port** is taken from the bridge forwarding table, so it is the port the + traffic actually arrives on, not a guess from topology. + +The resulting warning names the mechanism: + +> **This change is predicted to cut management access to 2GT-NW-100G.** +> Management arrives untagged on `sfp28-1` (PVID 1) — the gateway's MAC is learned there — +> but VLAN 1 has no bridge VLAN entry listing `bridge1` as an untagged member. + +### Overriding a verdict + +Deliberately awkward. The API requires `confirm_lockout: true` in the request body, and +the UI requires typing the device name. The change then runs under Change Guard regardless, +so an override is a decision to *rely on* auto-revert — not to bypass protection. + +A pre-existing violation is reported separately from one your change would cause. That +distinction matters: if an invariant is already broken, that check cannot detect a new +break, and the verdict says so rather than implying a clean result. + +### Capability probe + +``` +POST /api/devices/:id/change-guard/probe +``` + +Reports what safety mechanisms a device actually supports, tested on the device rather +than assumed. Non-destructive — it creates and removes a harmless scheduler and leaves +nothing behind. + +It also establishes whether `/system history` can revert a change without a reboot. +Measured on RouterOS 7.23 over the binary API: + +| Command | Result | +|---|---| +| `/system/history/print` | works | +| `/system/history/undo` | no such command | +| `/undo numbers=` | unknown parameter | +| `/undo` | works — reverts the newest action, no reboot | + +Undo is therefore real but **untargeted**, and it cannot replace auto-revert: a dead-man +switch has to fire once the manager has already lost contact, and issuing an undo requires +the contact that was just lost. Binary restore remains the default. + +## Config Health + +A scheduled, read-only audit for configurations RouterOS accepts without complaint and +then quietly fails to honour. Each finding explains what it does to the network, how to +fix it, how long it has been present, and links the relevant MikroTik documentation. + +Findings appear on the device's Security tab and in the dashboard's *Things to handle*. + +### What it checks + +| Finding | Consequence | +|---|---| +| IP address on a bridge slave port | The address is served by the bridge, or not at all | +| VLAN interface on a bridge slave port | The interface receives nothing; management may be unreachable | +| VLAN interface added as a bridge port | Forwarding loop; STP ports flap | +| Bond slave that is also a bridge port | Undefined forwarding path; the bond silently loses members | +| VLAN interface whose VLAN the bridge is not tagged in | The interface is up and carries nothing | +| Management surviving only on a dynamic VLAN entry | Works today; disappears the moment a port admits only tagged frames | +| Port in no VLAN on a filtering bridge | Links up, then every frame is dropped | +| Multi-VLAN bridge entry with untagged ports | Ambiguous; RouterOS warns and applies it anyway | +| PVID next to `frame-type=admit-only-vlan-tagged` | The PVID is stored but never applied | +| Several bridges competing for hardware offload | One silently falls back to CPU forwarding | +| MTU above L2MTU | Large frames dropped without error | +| Duplicate address on two interfaces | Only one can answer; which is not stated by the config | + +### Settings + +| Setting | Default | Description | +|---|---|---| +| `config_health_enabled` | `true` | Runs the standing audit | +| `config_health_interval_min` | `60` | Cadence per device — one read-only snapshot over the API | + +### A note on false positives + +Rules are calibrated against real hardware rather than documentation alone. For example, +RouterOS 7 relocates an address configured on a bridge slave port onto the bridge itself +(`actual-interface`), so that stock configuration works — and is reported as informational +rather than critical. Similarly, a port that belongs to no VLAN is only reported when it +is actually active, because an unplugged port carrying no configuration is not a fault. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..55d9811 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,76 @@ +# Configuration + +[← Documentation index](README.md) + +## Environment variables + +Set in `.env` at the project root. Changing any of these requires a container restart. + +| Variable | Default | Description | +|---|---|---| +| `JWT_SECRET` | *auto-generated & persisted* | Signs session tokens. Set it to pin your own value. | +| `ENCRYPTION_KEY` | *auto-generated & persisted* | Encrypts device passwords at rest. Set it to pin your own value. | +| `CORS_ORIGIN` | *localhost defaults* | Comma-separated browser origins allowed to call the API. **Required in production.** | +| `DB_PASSWORD` | `mikrotik_secure_pw` | PostgreSQL password | +| `INFLUXDB_TOKEN` | `mytoken123456789` | InfluxDB admin token | +| `INFLUXDB_ORG` | `mikrotik-manager` | InfluxDB organization | +| `INFLUXDB_BUCKET` | `metrics` | InfluxDB bucket for time-series data | +| `INFLUXDB_ADMIN_PASSWORD` | `admin_password_123` | InfluxDB admin UI password | +| `HTTP_PORT` | `80` | Host port for HTTP (redirects to HTTPS) | +| `HTTPS_PORT` | `443` | Host port for HTTPS | + +> Never commit `.env`. It is already listed in `.gitignore`. + +For a production deployment the only variable you genuinely must set is `CORS_ORIGIN`. +Both secrets generate themselves safely — see below. + +## Secret management (self-healing) + +`JWT_SECRET` and `ENCRYPTION_KEY` are managed automatically so that a fresh install is +secure by default and an upgrade never strands existing data. + +- **Set to a strong value in the environment** — that value is used, and you stay in + control. +- **Unset, or left at an old default** — the backend generates a strong secret once, + persists it to the `app_data` volume (`SECRETS_DIR`, default `/app/data`), and reuses + it on every boot. + +Secrets live outside the database deliberately: a database dump alone cannot reveal the +key protecting the credentials stored inside it. + +### Upgrading is non-breaking + +Existing ciphertext is decrypted through a **legacy-key fallback** — including previous +shipped defaults — and transparently re-encrypted under the current key by a background +sweep. Nothing needs to be re-entered. + +Rotating the JWT secret off a public default invalidates sessions signed with it. Users +simply log in again once. + +### Key rotation + +Set a new `ENCRYPTION_KEY` (or delete the persisted secret to force regeneration) and +restart. Old rows keep decrypting through the fallback and are re-encrypted forward +automatically. + +> **If the persisted secret and every prior key are lost**, ciphertext encrypted under +> that key cannot be recovered. Re-enter device credentials, or restore from a backup. + +## Settings stored in the database + +These are edited in the Settings UI and take effect without a restart: + +| Area | Settings | Documented in | +|---|---|---| +| Change Guard | `change_guard_enabled`, `change_guard_mode`, `change_guard_timeout_sec` | [Change Guard](change-guard.md) | +| Config Health | `config_health_enabled`, `config_health_interval_min` | [Change Guard](change-guard.md) | +| Polling | fast / slow / log intervals, MAC scan, spectral and AP scan cadence | Settings → Polling | +| Config snapshots | `config_snapshot_enabled`, `config_snapshot_interval_min`, `config_snapshot_retention` | Settings → Config History | +| NetFlow | collector address and port, version, retention, top-N clients | Settings → NetFlow | +| Alerting | rules, thresholds, cooldowns, channels | [Alerting](alerting.md) | + +## TLS + +A self-signed certificate is generated on first run. Replace it under +**Settings → TLS Certificate** by uploading a certificate and private key; nginx +terminates TLS and redirects HTTP to HTTPS. diff --git a/docs/sso-oidc.md b/docs/sso-oidc.md new file mode 100644 index 0000000..26263f9 --- /dev/null +++ b/docs/sso-oidc.md @@ -0,0 +1,71 @@ +# Single sign-on (OIDC) + +[← Documentation index](README.md) + +Authenticate against any standards-compliant OpenID Connect provider — Microsoft Entra ID, +Okta, Google Workspace, Keycloak, Authentik and others. + +Authorization Code flow with PKCE. The server validates the ID token (JWKS signature plus +`iss`, `aud`, `exp` and `nonce`) and then issues the platform's normal session, so SSO +users get the same experience as local ones. + +**No environment variables and no redeploy** — everything is configured in the UI, with +the client secret encrypted at rest. + +## 1. Register the application with your provider + +Create an OIDC/OAuth **web application** and note: + +- **Issuer URL** — the base that serves `/.well-known/openid-configuration` +- **Client ID** +- **Client Secret** — optional; public clients with no secret work via PKCE + +Set the redirect / callback URI to: + +``` +https:///api/auth/oidc/callback +``` + +The exact value is shown in the settings panel, which is worth copying from rather than +typing. + +## 2. Configure it in the app + +As an admin, go to **Settings → SSO / OIDC**. + +| Field | Purpose | +|---|---| +| **Issuer URL** | Your provider's base URL. **Test discovery** verifies it is reachable before you save. | +| **Client ID / Secret** | From step 1. The secret is write-only — leave it blank to keep the existing one. | +| **Scopes** | Default `openid profile email`. Add a groups scope if your IdP needs one to emit group membership. | +| **Username claim** | Default `preferred_username` | +| **Email claim** | Default `email` | +| **Groups claim** | Default `groups` | +| **Group → role mapping** | Map IdP groups to admin / operator / viewer, e.g. `net-admins → admin`. Highest-privilege match wins. | +| **Default role** | For users matching no mapping. Defaults to **viewer**. | +| **Auto-provision** | Create a local user automatically on first sign-in. | +| **Link by verified email** | Attach an SSO identity to an existing local account when the IdP's `email_verified` matches. | +| **Allowed email domains** | Optional allowlist restricting who may sign in. | +| **Public base URL** | Pin the redirect URI when your external URL differs from what the backend sees. | +| **Button label** | Text on the login button, e.g. "Sign in with Okta". | + +Enable it and save. A **Sign in with SSO** button appears on the login page. + +## Behaviour and safety + +**Provisioning and linking.** First sign-in creates a user when auto-provision is on, or +links to an existing local account when `email_verified` matches — no duplicate account. +SSO users have no local password and are labelled with an **SSO** badge under +Users & Roles. + +**Roles are resolved server-side** from your group mapping on every login. Existing users +are never silently demoted when a login carries no matching group, so a temporary IdP +misconfiguration cannot strip your admins. + +**Break-glass.** Local username and password login stays available. The seeded local admin +can always get in, even if the IdP is misconfigured, expired, or unreachable. Do not delete +that account. + +**Behind a reverse proxy.** The redirect URI is derived from the request host. If your +external URL differs from what the backend sees — a proxy, a custom domain, a tunnel — +set **Public base URL** to pin it, or the provider will reject the callback. diff --git a/frontend/package.json b/frontend/package.json index e4089cc..0d46736 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,6 +1,6 @@ { "name": "mikrotik-manager-frontend", - "version": "0.23.3-beta", + "version": "0.23.4-beta", "private": true, "type": "module", "scripts": { diff --git a/frontend/src/version.ts b/frontend/src/version.ts index 856aa47..f597070 100644 --- a/frontend/src/version.ts +++ b/frontend/src/version.ts @@ -1 +1 @@ -export const APP_VERSION = 'v0.23.3 Beta'; +export const APP_VERSION = 'v0.23.4 Beta';