diff --git a/README.md b/README.md index 57357e3..6448570 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@

- Version + Version License Docker TypeScript @@ -33,7 +33,7 @@ - [Visibility and troubleshooting](#visibility-and-troubleshooting) - [Fleet operations](#fleet-operations) - [Access control and platform](#access-control-and-platform) -- [Reference](#reference) +- [Documentation](#documentation) - [Contributing](#contributing) - [License](#license) @@ -259,28 +259,9 @@ The layers described [above](#what-makes-it-different), plus the surrounding mac - **Capability probe** — `POST /api/devices/:id/change-guard/probe` reports what safety mechanisms a device actually supports, tested rather than assumed, and leaves nothing behind - **Guard history** — every protected change recorded per device as `committed`, `reverted`, or `failed` -

-What you see when auto-revert fires - -
- -The request returns **HTTP 200** with `guard.auto_reverting: true` and *"Contact with the device was -lost while applying this change. It is restoring itself and should come back shortly."* — not an error. - -This matters: the most dangerous changes sever the very connection carrying them, so the API call times -out even though the change succeeded on the device. **Reachability, not the exception, decides the -outcome.** A change that throws while the device is still reachable is a genuine failure, and the guard -is disarmed immediately so nothing reboots for nothing. - -Two consequences worth knowing before relying on it: - -- **Binary mode reboots the device** to restore. That is the price of a restore that recovers deleted - interfaces exactly. Script mode (`/export` + `/import`) avoids the reboot but is additive, so it - cannot recover a deletion. -- **The restore point is saved before the scheduler is armed**, so a rollback returns a configuration - containing neither the scheduler nor the change. Self-cleaning, and a revert loop is impossible. - -
+For the full behaviour — what the API returns when contact is lost, how binary and script +modes differ, and how to choose a timeout — see +[Change Guard and Config Health](docs/change-guard.md). ### Monitoring and operations @@ -306,7 +287,7 @@ Two consequences worth knowing before relying on it: - **Spectral and AP scans** — scheduled or on demand, per radio - **Rogue & neighbour AP detection** — stored scan results cross-referenced against your own SSIDs and radio MACs; a foreign BSSID broadcasting your SSID is flagged as an evil twin - **Device fingerprinting** — every client classified into a category (server, computer, phone, TV, camera, printer, console, IoT…) from OUI and hostname, overridable per client and persistent across polls -- **CAPsMAN** — role detection (standalone / CAP / controller / controller-with-local-radios), provisioned interfaces labelled with their controller, and a controller panel grouping every AP with its radios, live channel, provisioned SSID and client counts. CAPs are matched to managed devices by radio MAC — hardware identity, so it cannot confuse two devices on two segments. Read-only for now; see [CAPsMAN scope](#capsman-scope) +- **CAPsMAN** — role detection (standalone / CAP / controller / controller-with-local-radios), provisioned interfaces labelled with their controller, and a controller panel grouping every AP with its radios, live channel, provisioned SSID and client counts. CAPs are matched to managed devices by radio MAC — hardware identity, so it cannot confuse two devices on two segments. Read-only for now; see [CAPsMAN scope](docs/capsman.md#current-limits) - **Guest WiFi** — a guided wizard that builds a full captive portal in one pass: guest SSID on every radio, VLAN segregation, IP pool, DHCP, hotspot profile, bandwidth-limited user profile and optional NAT. Plus batch vouchers with printable 3-up sheets, a live guests-online table, and an inline walled garden ### Firewall and network services @@ -367,120 +348,30 @@ Each network service supports multi-device management with conflict detection: - **Roles** — Admin, Operator (read/write), Viewer (read-only) - **Two-factor authentication** — per-user TOTP via QR code, required at login once enabled -- **Single sign-on (OIDC)** — any standards-compliant provider (Entra ID, Okta, Google Workspace, Keycloak, Authentik…), with group-to-role mapping, auto-provisioning, and local login retained as break-glass. Configured entirely in the UI — see [Single sign-on setup](#single-sign-on-oidc) +- **Single sign-on (OIDC)** — any standards-compliant provider (Entra ID, Okta, Google Workspace, Keycloak, Authentik…), with group-to-role mapping, auto-provisioning, and local login retained as break-glass. Configured entirely in the UI — see [the SSO guide](docs/sso-oidc.md) - **Scoped API tokens** — `mtm_…` tokens with read or write scope and optional expiry, shown once and stored only as a SHA-256 hash. No token can perform admin actions or manage other tokens - **Outbound webhooks** — subscribe to any of twelve events, delivered as HMAC-SHA256-signed JSON POSTs through the same pipeline that respects alert rules, cooldowns and maintenance windows - **Scheduled email reports** — daily, weekly or monthly HTML fleet summaries to any recipient list - **Credential presets** — shared device credentials, optionally restricted to admins only -- **Encryption at rest** — device passwords encrypted with AES-256-GCM under a [self-healing key](#secret-management-self-healing) +- **Encryption at rest** — device passwords encrypted with AES-256-GCM under a [self-healing key](docs/configuration.md#secret-management-self-healing) - **TLS** — self-signed certificate generated on first run, replaceable via the Settings UI; nginx terminates TLS and redirects HTTP --- -## Reference +## Documentation -### Environment variables +Deeper reference material lives in [`docs/`](docs/README.md), versioned alongside the code +so it moves when behaviour does. -Set in `.env`. Both secrets auto-generate on first boot if you leave them unset. - -| Variable | Default | Description | -|---|---|---| -| `JWT_SECRET` | *auto-generated & persisted* | Signs session tokens. Set it to pin your own value. | -| `ENCRYPTION_KEY` | *auto-generated & persisted* | Encrypts device passwords at rest. Set it to pin your own value. | -| `CORS_ORIGIN` | *localhost defaults* | Comma-separated browser origins allowed to call the API. **Required in production.** | -| `DB_PASSWORD` | `mikrotik_secure_pw` | PostgreSQL password | -| `INFLUXDB_TOKEN` | `mytoken123456789` | InfluxDB admin token | -| `INFLUXDB_ORG` | `mikrotik-manager` | InfluxDB organization | -| `INFLUXDB_BUCKET` | `metrics` | InfluxDB bucket for time-series data | -| `INFLUXDB_ADMIN_PASSWORD` | `admin_password_123` | InfluxDB admin UI password | -| `HTTP_PORT` | `80` | Host port for HTTP (redirects to HTTPS) | -| `HTTPS_PORT` | `443` | Host port for HTTPS | - -> Never commit `.env`. It is already in `.gitignore`. - -### Change Guard and Config Health settings - -These live in the Settings UI rather than `.env`, so they apply without a restart. - -| Setting | Default | Description | -|---|---|---| -| `change_guard_enabled` | `true` | Master switch. When off, guarded changes still apply — with no safety net and no lockout refusal. | -| `change_guard_mode` | `binary` | `binary` = `/system backup` restore; recovers deletions exactly, **reboots the device**. `script` = `/export` + `/import`; no reboot, but additive, so it cannot undo a deletion. | -| `change_guard_timeout_sec` | `120` | How long the device waits before rescuing itself. Must exceed the manager's verification window (~40s) with margin for a slow device. | -| `config_health_enabled` | `true` | Runs the standing configuration audit. | -| `config_health_interval_min` | `60` | Audit cadence per device — one read-only snapshot over the API. | - -Overriding a predicted lockout is deliberate by design: the API requires `confirm_lockout: true` and the -UI requires typing the device name. The change then runs under Change Guard regardless, so an override -is a decision to *rely on* auto-revert — not to bypass protection. - -### CAPsMAN scope - -Provisioning, configuration edits and interface enable/disable are **deliberately not exposed**. A -CAPsMAN configuration applies to every access point bound to it simultaneously, which is the fleet-wide -version of the failure Change Guard exists to prevent. Those operations are held back until they carry -the same prediction-and-revert protection that per-device changes do. - -Legacy `/caps-man` (the pre-RouterOS 7 controller) is a separate API tree and is not covered yet. - -### Secret management (self-healing) - -`JWT_SECRET` and `ENCRYPTION_KEY` are managed automatically: - -- **Set to a strong value in the environment** — that value is used, and you stay in control. -- **Unset or left at an old default** — the backend generates a strong secret once, persists it to the - `app_data` volume (`SECRETS_DIR`, default `/app/data`), and reuses it every boot. Secrets live outside - the database, so a database dump alone cannot reveal the key protecting the credentials inside it. - -Upgrades are non-breaking. Existing ciphertext is decrypted via a **legacy-key fallback** (including -previous defaults) and transparently re-encrypted under the current key by a background sweep. Rotating -the JWT secret off a public default invalidates sessions signed with it — users simply log in again. - -**Key rotation:** set a new `ENCRYPTION_KEY` (or delete the persisted secret to force regeneration) and -restart. Old rows keep decrypting via the fallback and are re-encrypted forward automatically. **If the -persisted secret and every prior key are lost**, ciphertext under that key cannot be recovered — re-enter -device credentials or restore a backup. - -### Single sign-on (OIDC) - -Authorization Code flow with PKCE. The server validates the ID token (JWKS signature plus `iss`, `aud`, -`exp`, `nonce`) and then issues the platform's normal session, so SSO users get the same experience as -local ones. **No environment variables or redeploy** — everything is configured in the UI, with the -client secret encrypted at rest. - -**1. Register the app with your provider.** Create an OIDC web application and note its **Issuer URL** -(the base serving `/.well-known/openid-configuration`), **Client ID** and **Client Secret** — public -clients with no secret work too, via PKCE. Use this redirect URI: - -``` -https:///api/auth/oidc/callback -``` - -**2. Configure it under Settings → SSO / OIDC.** - -| Field | Purpose | +| Document | Covers | |---|---| -| **Issuer URL** | Your provider's base URL; **Test discovery** verifies reachability. | -| **Client ID / Secret** | From step 1. The secret is write-only; leave blank to keep the existing one. | -| **Scopes** | Default `openid profile email`. Add a groups scope if your IdP needs one. | -| **Username / Email / Groups claim** | Defaults `preferred_username`, `email`, `groups`. | -| **Group → role mapping** | Map IdP groups to admin / operator / viewer. Highest-privilege match wins. | -| **Default role** | For users matching no mapping (default **viewer**). | -| **Auto-provision** | Create a local user on first sign-in. | -| **Link by verified email** | Attach an SSO identity to an existing account when `email_verified` matches. | -| **Allowed email domains** | Optional allowlist. | -| **Button label** | Text on the login button, e.g. "Sign in with Okta". | - -**Behaviour and safety:** - -- **Provisioning** — first sign-in creates a user, or links to an existing local account when the - verified email matches. SSO users have no local password. -- **Roles** are always resolved server-side from your mapping, and existing users are never silently - demoted when a login carries no matching group. -- **Break-glass** — local login stays available, so the seeded admin can always get in even if the IdP - is misconfigured or unreachable. -- **Behind a proxy** — the redirect URI derives from the request host; set **Public base URL** to pin it - when your external URL differs from what the backend sees. +| [Configuration](docs/configuration.md) | Environment variables, secret management, key rotation, TLS | +| [Change Guard and Config Health](docs/change-guard.md) | How the safety system works, its settings, and what to expect when it fires | +| [Alerting](docs/alerting.md) | Rules, channels, ntfy priority mapping, webhooks, maintenance windows | +| [Single sign-on (OIDC)](docs/sso-oidc.md) | Identity provider setup, group-to-role mapping, break-glass behaviour | +| [CAPsMAN](docs/capsman.md) | How centrally provisioned APs are modelled, and current limits | +| [API and automation](docs/api.md) | Scoped tokens, preflighting a change, driving the platform from scripts | +| [Architecture](docs/architecture.md) | Data stores, polling model, and where the interesting logic lives | ### Tech stack @@ -489,45 +380,11 @@ https:///api/auth/oidc/callback | **Frontend** | React 18, TypeScript, Vite, Tailwind CSS | | **State / data** | TanStack Query v5, React Router v6, Zustand | | **Charts / topology / maps** | Recharts, @xyflow/react, Leaflet | -| **Terminal** | xterm.js | | **Backend** | Node.js, Express, TypeScript | -| **Primary DB** | PostgreSQL 15 | -| **Time-series DB** | InfluxDB 2.7 | -| **Cache / queue** | Redis 7, BullMQ | +| **Data stores** | PostgreSQL 15, InfluxDB 2.7, Redis 7 + BullMQ | | **Real-time** | Socket.IO | | **Device comms** | RouterOS API (8728 / 8729), SSH2 | -| **Proxy** | nginx — TLS termination and static serving | -| **Container** | Docker Compose | - -### Project structure - -``` -mikrotik-manager/ -├── frontend/ # React + TypeScript (Vite) -│ └── src/ -│ ├── pages/ # One component per route -│ ├── components/ # Shared UI -│ ├── services/ # API client (Axios) -│ ├── hooks/ # Custom React hooks -│ └── types/ # TypeScript definitions -│ -├── backend/ # Node.js + Express + TypeScript -│ └── src/ -│ ├── routes/ # REST API handlers -│ ├── services/ # Polling, alerts, backups, orchestration -│ │ ├── mikrotik/ # RouterOS API client and device collector -│ │ ├── changeGuard/# Auto-revert, lockout prediction, Config Health -│ │ └── topology/ # Neighbour graph construction -│ ├── db/ # Schema and migrations -│ ├── config/ # Database, InfluxDB, Redis connections -│ ├── middleware/ # Auth, audit logging, error handling -│ └── utils/ # Crypto, OUI lookup, VLAN parsing -│ -├── nginx/ # Reverse proxy config and Dockerfile -├── docker-compose.yml # Build from source -├── docker-compose.ghcr.yml # Pre-built images -└── .env.example -``` +| **Proxy / container** | nginx, Docker Compose | --- diff --git a/backend/package.json b/backend/package.json index 07d23be..2fbc3e3 100644 --- a/backend/package.json +++ b/backend/package.json @@ -1,6 +1,6 @@ { "name": "mikrotik-manager-backend", - "version": "0.23.3-beta", + "version": "0.23.4-beta", "description": "Mikrotik Network Management Platform - Backend API", "main": "dist/index.js", "scripts": { diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..fcf68a3 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,20 @@ +# MikroTik Manager documentation + +Reference material for people already running the platform. If you are still deciding +whether to try it, the [project README](../README.md) is the better starting point. + +| Document | Covers | +|---|---| +| [Configuration](configuration.md) | Environment variables, secret management and key rotation | +| [Change Guard and Config Health](change-guard.md) | How the safety system works, its settings, and what to expect when it fires | +| [Alerting](alerting.md) | Alert rules, delivery channels, webhooks, scheduled reports, maintenance windows | +| [Single sign-on (OIDC)](sso-oidc.md) | Identity provider setup, group-to-role mapping, break-glass behaviour | +| [CAPsMAN](capsman.md) | How centrally provisioned access points are modelled, and current limits | +| [API and automation](api.md) | Scoped tokens, authentication, and driving the platform from scripts | +| [Architecture](architecture.md) | How the system is put together, for contributors | + +## Conventions used here + +- **Settings in the UI** are stored in the database and apply without a restart. +- **Environment variables** live in `.env` and require a container restart to change. +- Anything described as *read-only* makes no writes to your devices. diff --git a/docs/alerting.md b/docs/alerting.md new file mode 100644 index 0000000..520f36a --- /dev/null +++ b/docs/alerting.md @@ -0,0 +1,81 @@ +# Alerting + +[← Documentation index](README.md) + +## Alert rules + +Each rule can be enabled independently, with its own threshold where applicable and a +cooldown that prevents a flapping device from flooding your channels. + +| Event | Notes | +|---|---| +| `device_offline` / `device_online` | Raised by the poller; recovery closes the outage record | +| `high_cpu` / `high_memory` | Configurable threshold | +| `cert_expiry` | Certificate approaching expiry | +| `firmware_update_available` | A newer RouterOS release exists for the device | +| `log_error` / `log_warning` | Matched from the device's own log | +| `device_discovered` | An unmanaged neighbour appeared via LLDP/CDP/MNDP | +| `config_drift` | The device's configuration changed (off by default) | + +Alerts are suppressed for devices inside an active [maintenance window](#maintenance-windows). + +## Delivery channels + +Email, Slack, Discord, Telegram and **ntfy**. Channels are configured under +**Settings → Alerts → Channels**; secrets are masked on read and preserved when you save +a channel without retyping them. + +### ntfy + +Works with the public `ntfy.sh` or your own instance. + +| Field | Notes | +|---|---| +| **Server URL** | Blank for `https://ntfy.sh`, or your own instance | +| **Topic** | Required | +| **Access token** | `tk_…`; preferred, since it can be scoped and revoked | +| **Username / password** | Basic-auth alternative | +| **Manager URL** | Optional — makes each notification tappable, opening the device it refers to | + +Authentication is optional, but an unprotected topic means anyone who guesses the name +can publish to it, so use one of the two. + +**Priority mapping.** Severity maps onto ntfy's 1–5 scale so the channel stays worth +being woken by: + +| Events | Priority | +|---|---| +| Device offline, log errors, CPU/memory pressure | **4** — breaks through do-not-disturb | +| Certificate expiry, config drift, firmware available | 3 — default | +| Device recovery, new device discovered | 2 — low | + +Each notification carries its event type as a **tag**, so a client can be filtered to +wake only for the events you care about rather than needing a channel per event. + +## Outbound webhooks + +Subscribe any URL to twelve events: device up/down, log errors, high CPU, high memory, +certificate expiry, device discovered, firmware update available, config drift, and +firmware rollout completed/failed. + +Deliveries are JSON `POST`s, **HMAC-SHA256 signed** in `X-MTM-Signature` when a secret is +set. Last-delivery status is tracked per webhook and there is a Send-test button. + +Webhooks fire through the same pipeline as other alerts, so they respect alert rules, +cooldowns and maintenance windows rather than bypassing them. + +## Scheduled email reports + +Daily, weekly or monthly HTML fleet summaries to any recipient list, using the same SMTP +settings as email alerts. Each report covers devices online, outages and total downtime, +error and warning counts, updates pending, backups taken, and top clients by traffic. +Send-now is available for an immediate copy. + +## Maintenance windows + +Schedule planned downtime per device, or across a group, so alerts are suppressed +automatically rather than being muted globally and forgotten. + +- One-time or recurring (cron-based) +- Active windows can be deactivated early +- Managed under **Settings → Maintenance** diff --git a/docs/api.md b/docs/api.md new file mode 100644 index 0000000..e546b32 --- /dev/null +++ b/docs/api.md @@ -0,0 +1,77 @@ +# API and automation + +[← Documentation index](README.md) + +The web UI is a client of the same REST API you can drive yourself. Anything the interface +does is available to a script. + +## Scoped API tokens + +Issue tokens under **Settings → Automation**. + +- Prefix `mtm_…`, with **read** or **write** scope and an optional expiry +- Shown **once** on creation — only a SHA-256 hash is stored +- Mapped onto the role model: no token can perform admin actions or manage other tokens, + regardless of scope + +```bash +curl -sk https://manager.example.com/api/devices \ + -H "Authorization: Bearer mtm_your_token_here" +``` + +A token with `read` scope is rejected on any mutating request, so a monitoring integration +cannot change your network even if the token leaks. + +## Session authentication + +The UI authenticates with a JWT obtained from `/api/auth/login`, optionally followed by +`/api/auth/totp/verify` when two-factor is enabled. Tokens are bearer credentials and are +sent in the same `Authorization` header. + +For scripting, prefer an API token — it is scoped, revocable, and unaffected by password +or 2FA changes. + +## Endpoints worth knowing + +| Endpoint | Purpose | +|---|---| +| `GET /api/devices` | Fleet inventory with status, model, versions and serials | +| `POST /api/devices/:id/sync` | Force an immediate full collection for one device | +| `GET /api/devices/:id/management-path` | How the manager reaches a device, with the reason for each hop | +| `POST /api/devices/:id/preflight` | Analyse a change **without applying it** | +| `POST /api/devices/:id/change-guard/probe` | Report which safety mechanisms the device supports | +| `GET /api/devices/:id/config-health` | Latest standing-audit findings | +| `GET /api/topology` | Graph of devices, links, external nodes, and distrusted identifiers | +| `GET /api/operations/insights` | The dashboard's "things to handle" feed | + +`preflight` is the useful one for automation: it returns the same verdict the UI shows, +so a pipeline can refuse its own change before touching the device. See +[Change Guard](change-guard.md#lockout-prediction). + +## Applying a guarded change from a script + +Guarded endpoints return **409** with `lockout: true` and a verdict when a change is +predicted to sever management. To proceed anyway, resend with `confirm_lockout: true` — +the change then runs under Change Guard, so a mistake still recovers itself. + +```bash +# Refused, with an explanation +curl -sk -X PUT https://manager.example.com/api/devices/8/ports/ether1/vlan \ + -H "Authorization: Bearer mtm_…" -H 'Content-Type: application/json' \ + -d '{"pvid":99,"tagged_vlans":[],"untagged_vlans":[99]}' + +# Accepted, relying on auto-revert +curl -sk -X PUT https://manager.example.com/api/devices/8/ports/ether1/vlan \ + -H "Authorization: Bearer mtm_…" -H 'Content-Type: application/json' \ + -d '{"pvid":99,"tagged_vlans":[],"untagged_vlans":[99],"confirm_lockout":true}' +``` + +A successful guarded change returns a `guard` block describing what happened — +`confirmed`, `auto_reverting`, or an `unprotected_reason` when the safety net could not be +armed. Check it rather than relying on the HTTP status alone. + +## Webhooks + +For push rather than poll, see [Alerting → outbound webhooks](alerting.md#outbound-webhooks). +Deliveries are HMAC-SHA256 signed and respect alert rules, cooldowns and maintenance +windows. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..8c33c2d --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,116 @@ +# Architecture + +[← Documentation index](README.md) + +For contributors, and for anyone wanting to know what happens between the UI and a device. + +## The shape of it + +``` +Browser ──► nginx ──► Express API ──► RouterOS API / SSH ──► devices + │ + ├──► PostgreSQL current state, config, users, audit + ├──► InfluxDB time-series metrics + └──► Redis job queue and locks +``` + +Three data stores, because they answer different questions: + +| Store | Holds | Why not the others | +|---|---|---| +| **PostgreSQL** | Devices, interfaces, VLANs, clients, users, audit log, findings | Relational, needs joins and constraints | +| **InfluxDB** | CPU, memory, traffic, client counts, RF quality | Time-series at poll cadence; would bloat Postgres | +| **Redis** | BullMQ job queues, per-device locks, poll gating | Ephemeral, needs atomic operations | + +## Polling + +A scheduler enqueues work per device on separate cadences, so a slow operation never +delays a fast one: + +| Cycle | Default | Collects | +|---|---|---| +| Fast | 30s | Reachability, CPU, memory, traffic counters, clients | +| Slow | 5min | Interfaces, VLANs, addresses, neighbours, STP, wireless, CAPsMAN | +| Logs | 60s | Device log entries, feeding log alerts | +| Config snapshot | 60min | `/export`, deduplicated by content hash | +| Config health | 60min | Read-only configuration audit | +| MAC / spectral / AP scan | configurable | Switch MAC tables, RF surveys | + +Devices are probed once for wireless capability and the result recorded, so non-wireless +devices are skipped cheaply on every later poll. + +## Device communication + +`RouterOSClient` speaks the RouterOS **binary API** (8728, or 8729 with TLS). It is used +for essentially everything, including the safety machinery, because many deployments never +configure SSH. + +SSH is used only where RouterOS offers no API equivalent — `/export` for config snapshots, +binary backup retrieval, packet capture download, and the in-browser terminal. + +## Where the interesting logic lives + +``` +backend/src/services/ +├── mikrotik/ +│ ├── RouterOSClient.ts binary protocol client +│ ├── DeviceCollector.ts all collection and device mutation +│ └── capsman.ts role detection, radio matching (pure) +├── changeGuard/ +│ ├── ChangeGuard.ts restore point, scheduler, verify, commit +│ ├── pathModel.ts live snapshot, management-path resolution (pure) +│ ├── invariants.ts what must hold for management to survive (pure) +│ ├── analyzeChange.ts simulate a change, diff the invariants (pure) +│ └── configHealth.ts standing audit rules (pure) +├── topology/ +│ └── buildTopology.ts neighbour graph construction (pure) +├── netflow/ NetFlow v9 / IPFIX collector and aggregator +└── PollerService.ts scheduling and workers +``` + +Anything marked *pure* is a function over a snapshot with no I/O. That is deliberate: this +logic is subtle, its failure mode is a plausible-looking wrong answer rather than a crash, +and it must be testable without the hardware it reasons about. Those modules carry the bulk +of the test suite. + +## Repository layout + +``` +mikrotik-manager/ +├── frontend/ # React + TypeScript (Vite) +│ └── src/ +│ ├── pages/ # One component per route +│ ├── components/ # Shared UI +│ ├── services/ # API client (Axios) +│ ├── hooks/ # Custom React hooks +│ └── types/ # TypeScript definitions +│ +├── backend/ # Node.js + Express + TypeScript +│ └── src/ +│ ├── routes/ # REST API handlers +│ ├── services/ # See above +│ ├── db/ # Schema and migrations +│ ├── config/ # Database, InfluxDB, Redis connections +│ ├── middleware/ # Auth, audit logging, error handling +│ └── utils/ # Crypto, OUI lookup, VLAN parsing +│ +├── docs/ # This documentation +├── nginx/ # Reverse proxy config and Dockerfile +├── scripts/ # CI preflight and tooling +├── docker-compose.yml # Build from source +├── docker-compose.ghcr.yml # Pre-built images +└── .env.example +``` + +## Database migrations + +A single idempotent SQL script in `backend/src/db/migrate.ts` runs on every backend start. +Statements are written to be safe to re-run — `CREATE TABLE IF NOT EXISTS`, +`ADD COLUMN IF NOT EXISTS`, and guarded constraint swaps — so upgrading is just starting +the new image. + +## Before you push + +`scripts/ci-preflight.sh` runs the same gates as CI — lint, type-check, build, tests and +production audits for both packages — in about 35 seconds. Running it locally is faster +than finding out from a red badge. diff --git a/docs/capsman.md b/docs/capsman.md new file mode 100644 index 0000000..fee0598 --- /dev/null +++ b/docs/capsman.md @@ -0,0 +1,84 @@ +# CAPsMAN + +[← Documentation index](README.md) + +An access point provisioned by CAPsMAN holds none of its own wireless configuration — +the controller owns it. Reading the AP directly returns blank SSID, security and band, +which makes a perfectly healthy access point look broken. + +The platform models the controller relationship instead. + +## Device roles + +Every device is classified each poll from `/interface/wifi/capsman` and +`/interface/wifi/cap`: + +| Role | Meaning | +|---|---| +| `standalone` | Has radios, manages them itself | +| `cap` | Radios are provisioned by a controller | +| `controller` | Runs CAPsMAN; has no radios of its own | +| `controller_cap` | Runs CAPsMAN **and** its own radios under it — a supported MikroTik arrangement | +| `none` | No wireless hardware | + +The role is detected, not inferred from `device_type`. That matters because a dedicated +controller is usually classified as a router, and `device_type` is a label a human typed +rather than a fact about the hardware. The Radios tab and the wireless section follow the +detected role. + +## How CAPs are matched to devices + +The controller enumerates every radio it manages — local and remote — through +`/interface/wifi/radio`, each with a `radio-mac`. Those MACs are matched against every +interface MAC in the fleet. + +MAC rather than IP address, deliberately: a MAC is hardware identity and unique +fleet-wide, whereas an address is only unique within a broadcast domain. An installation +with several segments reusing the same addressing would otherwise attribute a radio to +the wrong device entirely. + +Two refinements come from real deployments: + +- A controller **mirrors each CAP's interfaces locally**, so a CAP's radio MAC genuinely + appears on two devices. A radio the controller reports as `local: false` lives on a CAP + by definition, so the controller is never a valid answer for it. +- A radio MAC often differs from the device's interface MAC in the final octet, so lookup + falls back to a five-octet prefix while keeping the OUI exact. + +A CAP that is **not** in the fleet is surfaced as unmanaged rather than hidden — that is +usually the reason an access point appears to be missing. + +## What you get + +The **CAPsMAN panel** on the Wireless page lists every controller with the access points +it provisions, one row per radio: the AP it lives on (linked), interface name, provisioned +SSID, operating channel, connected clients, and state. + +Two details are read from the controller rather than the CAP, because the CAP does not +know them: + +- **SSID** — held on the controller's mirror of the interface. +- **Client counts** — a CAP's own registration table is empty when traffic is processed + centrally. Counts come from the registration table with each client attributed to its + radio by following `master-interface`, since clients register on the virtual AP carrying + the SSID rather than on the physical radio. + +## Write protection + +A provisioned radio is owned by the controller, so a local write to it is either rejected +outright or silently replaced at the next provision — while the API reports success. + +Five paths refuse or skip provisioned radios, naming the controller in the error: bulk +SSID deployment, interface create, interface edit, interface delete, and guest network +setup. + +## Current limits + +**Provisioning, configuration edits and interface enable/disable are deliberately not +exposed.** A CAPsMAN configuration applies to every access point bound to it +simultaneously — the fleet-wide version of the failure [Change Guard](change-guard.md) +exists to prevent. Those operations are held back until they carry the same +prediction-and-revert protection that per-device changes do. + +Legacy `/caps-man` — the pre-RouterOS 7 controller — is a separate API tree and is not +covered. The RouterOS 7 `wifi` stack is supported. diff --git a/docs/change-guard.md b/docs/change-guard.md new file mode 100644 index 0000000..e07c3e6 --- /dev/null +++ b/docs/change-guard.md @@ -0,0 +1,193 @@ +# Change Guard and Config Health + +[← Documentation index](README.md) + +RouterOS applies every command immediately and independently. There is no transaction, +no rollback, and no cross-object validation — so it will accept a change that severs the +very path you are managing the device over. MikroTik's own documented advice for enabling +VLAN filtering is to have a serial console ready. + +Three layers address that. This page covers how they behave and how to configure them; +the [README](../README.md#what-makes-it-different) has the short version. + +## Change Guard — device-side auto-revert + +Before applying a protected change: + +1. The device saves a **restore point** locally. +2. A **scheduler** is armed to reapply it after a timeout. +3. The change is applied. +4. Reachability is proven **on a brand-new connection** — not the one that made the change. +5. Confirmed → the scheduler is disarmed and the restore point deleted. + Unreachable → nothing is done, and the device restores itself. + +The restore point is saved *before* the scheduler is armed, so a rollback returns a +configuration containing neither the scheduler nor the change. It is self-cleaning, and +a revert loop is impossible. + +Everything runs over the RouterOS API, so it works on devices with no SSH credentials +configured. + +### Covered changes + +Twelve change types are guarded: + +| Area | Operations | +|---|---| +| Bridge | VLAN filtering toggle | +| Ports | PVID and tagged/untagged membership | +| Bridge VLANs | add, update, delete | +| Addressing | IP address add, remove | +| Routing | route add, remove | +| Bonding | bond create, delete | +| Services | management service enable/disable | + +### What you see when it fires + +Verified end to end on a CRS running RouterOS 7.23.3 by deliberately deleting a switch's +own management address: + +``` +T+0 restore point saved, revert scheduler armed, change applied +T+0s device drops off the network +T+40s manager gives up after 4 fresh connection attempts and reports back +T+120s device restores its own backup and reboots +T+3m device is back, change undone, scheduler and restore point gone +``` + +The request returns **HTTP 200** with `guard.auto_reverting: true` and the message +*"Contact with the device was lost while applying this change. It is restoring itself and +should come back shortly."* — deliberately not an error. + +This matters. The most dangerous changes sever the very connection carrying them, so the +API call times out even though the change succeeded on the device. **Reachability, not the +exception, decides the outcome.** A change that throws while the device is still reachable +is a genuine failure, and the guard is disarmed immediately so nothing reboots for nothing. + +### Guard history + +Every protected change is recorded per device: + +| Status | Meaning | +|---|---| +| `committed` | Change applied, device confirmed reachable, guard disarmed | +| `reverted` | Contact lost; the device was left to restore itself | +| `failed` | Change rejected by the device; nothing applied, guard disarmed | + +### Settings + +| Setting | Default | Description | +|---|---|---| +| `change_guard_enabled` | `true` | Master switch. When off, guarded changes still apply — with no safety net and no lockout refusal. | +| `change_guard_mode` | `binary` | See below. | +| `change_guard_timeout_sec` | `120` | How long the device waits before rescuing itself. | + +**Choosing a mode:** + +| Mode | Mechanism | Reboots? | Recovers a deletion? | +|---|---|---|---| +| `binary` | `/system backup` restore | **Yes** | Yes — restores the configuration exactly | +| `script` | `/export` then `/import` | No | **No** — `/import` is additive | + +`binary` is the default because a change that deletes an interface can only be undone by +a restore that recreates it. `script` avoids the reboot but cannot undo a deletion, which +is the case most likely to lock you out. + +**Choosing a timeout:** it must exceed the time the manager needs to verify reachability — +roughly 40 seconds across four connection attempts — with margin for a slow device. Too +short and a healthy device reverts a good change; too long and an outage lasts longer than +it needs to. + +## Lockout prediction + +Rather than blocklisting operations someone once decided were dangerous, the platform +reads live device state, resolves how the manager actually reaches the device, simulates +the proposed change against that model, and reports any invariant that flips from +satisfied to violated. + +Two details make the result precise rather than merely cautious: + +- The **manager's own address**, as the device sees it, is read from the device's + connection tracking — rather than being treated as unknowable behind NAT. +- The **ingress port** is taken from the bridge forwarding table, so it is the port the + traffic actually arrives on, not a guess from topology. + +The resulting warning names the mechanism: + +> **This change is predicted to cut management access to 2GT-NW-100G.** +> Management arrives untagged on `sfp28-1` (PVID 1) — the gateway's MAC is learned there — +> but VLAN 1 has no bridge VLAN entry listing `bridge1` as an untagged member. + +### Overriding a verdict + +Deliberately awkward. The API requires `confirm_lockout: true` in the request body, and +the UI requires typing the device name. The change then runs under Change Guard regardless, +so an override is a decision to *rely on* auto-revert — not to bypass protection. + +A pre-existing violation is reported separately from one your change would cause. That +distinction matters: if an invariant is already broken, that check cannot detect a new +break, and the verdict says so rather than implying a clean result. + +### Capability probe + +``` +POST /api/devices/:id/change-guard/probe +``` + +Reports what safety mechanisms a device actually supports, tested on the device rather +than assumed. Non-destructive — it creates and removes a harmless scheduler and leaves +nothing behind. + +It also establishes whether `/system history` can revert a change without a reboot. +Measured on RouterOS 7.23 over the binary API: + +| Command | Result | +|---|---| +| `/system/history/print` | works | +| `/system/history/undo` | no such command | +| `/undo numbers=` | unknown parameter | +| `/undo` | works — reverts the newest action, no reboot | + +Undo is therefore real but **untargeted**, and it cannot replace auto-revert: a dead-man +switch has to fire once the manager has already lost contact, and issuing an undo requires +the contact that was just lost. Binary restore remains the default. + +## Config Health + +A scheduled, read-only audit for configurations RouterOS accepts without complaint and +then quietly fails to honour. Each finding explains what it does to the network, how to +fix it, how long it has been present, and links the relevant MikroTik documentation. + +Findings appear on the device's Security tab and in the dashboard's *Things to handle*. + +### What it checks + +| Finding | Consequence | +|---|---| +| IP address on a bridge slave port | The address is served by the bridge, or not at all | +| VLAN interface on a bridge slave port | The interface receives nothing; management may be unreachable | +| VLAN interface added as a bridge port | Forwarding loop; STP ports flap | +| Bond slave that is also a bridge port | Undefined forwarding path; the bond silently loses members | +| VLAN interface whose VLAN the bridge is not tagged in | The interface is up and carries nothing | +| Management surviving only on a dynamic VLAN entry | Works today; disappears the moment a port admits only tagged frames | +| Port in no VLAN on a filtering bridge | Links up, then every frame is dropped | +| Multi-VLAN bridge entry with untagged ports | Ambiguous; RouterOS warns and applies it anyway | +| PVID next to `frame-type=admit-only-vlan-tagged` | The PVID is stored but never applied | +| Several bridges competing for hardware offload | One silently falls back to CPU forwarding | +| MTU above L2MTU | Large frames dropped without error | +| Duplicate address on two interfaces | Only one can answer; which is not stated by the config | + +### Settings + +| Setting | Default | Description | +|---|---|---| +| `config_health_enabled` | `true` | Runs the standing audit | +| `config_health_interval_min` | `60` | Cadence per device — one read-only snapshot over the API | + +### A note on false positives + +Rules are calibrated against real hardware rather than documentation alone. For example, +RouterOS 7 relocates an address configured on a bridge slave port onto the bridge itself +(`actual-interface`), so that stock configuration works — and is reported as informational +rather than critical. Similarly, a port that belongs to no VLAN is only reported when it +is actually active, because an unplugged port carrying no configuration is not a fault. diff --git a/docs/configuration.md b/docs/configuration.md new file mode 100644 index 0000000..55d9811 --- /dev/null +++ b/docs/configuration.md @@ -0,0 +1,76 @@ +# Configuration + +[← Documentation index](README.md) + +## Environment variables + +Set in `.env` at the project root. Changing any of these requires a container restart. + +| Variable | Default | Description | +|---|---|---| +| `JWT_SECRET` | *auto-generated & persisted* | Signs session tokens. Set it to pin your own value. | +| `ENCRYPTION_KEY` | *auto-generated & persisted* | Encrypts device passwords at rest. Set it to pin your own value. | +| `CORS_ORIGIN` | *localhost defaults* | Comma-separated browser origins allowed to call the API. **Required in production.** | +| `DB_PASSWORD` | `mikrotik_secure_pw` | PostgreSQL password | +| `INFLUXDB_TOKEN` | `mytoken123456789` | InfluxDB admin token | +| `INFLUXDB_ORG` | `mikrotik-manager` | InfluxDB organization | +| `INFLUXDB_BUCKET` | `metrics` | InfluxDB bucket for time-series data | +| `INFLUXDB_ADMIN_PASSWORD` | `admin_password_123` | InfluxDB admin UI password | +| `HTTP_PORT` | `80` | Host port for HTTP (redirects to HTTPS) | +| `HTTPS_PORT` | `443` | Host port for HTTPS | + +> Never commit `.env`. It is already listed in `.gitignore`. + +For a production deployment the only variable you genuinely must set is `CORS_ORIGIN`. +Both secrets generate themselves safely — see below. + +## Secret management (self-healing) + +`JWT_SECRET` and `ENCRYPTION_KEY` are managed automatically so that a fresh install is +secure by default and an upgrade never strands existing data. + +- **Set to a strong value in the environment** — that value is used, and you stay in + control. +- **Unset, or left at an old default** — the backend generates a strong secret once, + persists it to the `app_data` volume (`SECRETS_DIR`, default `/app/data`), and reuses + it on every boot. + +Secrets live outside the database deliberately: a database dump alone cannot reveal the +key protecting the credentials stored inside it. + +### Upgrading is non-breaking + +Existing ciphertext is decrypted through a **legacy-key fallback** — including previous +shipped defaults — and transparently re-encrypted under the current key by a background +sweep. Nothing needs to be re-entered. + +Rotating the JWT secret off a public default invalidates sessions signed with it. Users +simply log in again once. + +### Key rotation + +Set a new `ENCRYPTION_KEY` (or delete the persisted secret to force regeneration) and +restart. Old rows keep decrypting through the fallback and are re-encrypted forward +automatically. + +> **If the persisted secret and every prior key are lost**, ciphertext encrypted under +> that key cannot be recovered. Re-enter device credentials, or restore from a backup. + +## Settings stored in the database + +These are edited in the Settings UI and take effect without a restart: + +| Area | Settings | Documented in | +|---|---|---| +| Change Guard | `change_guard_enabled`, `change_guard_mode`, `change_guard_timeout_sec` | [Change Guard](change-guard.md) | +| Config Health | `config_health_enabled`, `config_health_interval_min` | [Change Guard](change-guard.md) | +| Polling | fast / slow / log intervals, MAC scan, spectral and AP scan cadence | Settings → Polling | +| Config snapshots | `config_snapshot_enabled`, `config_snapshot_interval_min`, `config_snapshot_retention` | Settings → Config History | +| NetFlow | collector address and port, version, retention, top-N clients | Settings → NetFlow | +| Alerting | rules, thresholds, cooldowns, channels | [Alerting](alerting.md) | + +## TLS + +A self-signed certificate is generated on first run. Replace it under +**Settings → TLS Certificate** by uploading a certificate and private key; nginx +terminates TLS and redirects HTTP to HTTPS. diff --git a/docs/sso-oidc.md b/docs/sso-oidc.md new file mode 100644 index 0000000..26263f9 --- /dev/null +++ b/docs/sso-oidc.md @@ -0,0 +1,71 @@ +# Single sign-on (OIDC) + +[← Documentation index](README.md) + +Authenticate against any standards-compliant OpenID Connect provider — Microsoft Entra ID, +Okta, Google Workspace, Keycloak, Authentik and others. + +Authorization Code flow with PKCE. The server validates the ID token (JWKS signature plus +`iss`, `aud`, `exp` and `nonce`) and then issues the platform's normal session, so SSO +users get the same experience as local ones. + +**No environment variables and no redeploy** — everything is configured in the UI, with +the client secret encrypted at rest. + +## 1. Register the application with your provider + +Create an OIDC/OAuth **web application** and note: + +- **Issuer URL** — the base that serves `/.well-known/openid-configuration` +- **Client ID** +- **Client Secret** — optional; public clients with no secret work via PKCE + +Set the redirect / callback URI to: + +``` +https:///api/auth/oidc/callback +``` + +The exact value is shown in the settings panel, which is worth copying from rather than +typing. + +## 2. Configure it in the app + +As an admin, go to **Settings → SSO / OIDC**. + +| Field | Purpose | +|---|---| +| **Issuer URL** | Your provider's base URL. **Test discovery** verifies it is reachable before you save. | +| **Client ID / Secret** | From step 1. The secret is write-only — leave it blank to keep the existing one. | +| **Scopes** | Default `openid profile email`. Add a groups scope if your IdP needs one to emit group membership. | +| **Username claim** | Default `preferred_username` | +| **Email claim** | Default `email` | +| **Groups claim** | Default `groups` | +| **Group → role mapping** | Map IdP groups to admin / operator / viewer, e.g. `net-admins → admin`. Highest-privilege match wins. | +| **Default role** | For users matching no mapping. Defaults to **viewer**. | +| **Auto-provision** | Create a local user automatically on first sign-in. | +| **Link by verified email** | Attach an SSO identity to an existing local account when the IdP's `email_verified` matches. | +| **Allowed email domains** | Optional allowlist restricting who may sign in. | +| **Public base URL** | Pin the redirect URI when your external URL differs from what the backend sees. | +| **Button label** | Text on the login button, e.g. "Sign in with Okta". | + +Enable it and save. A **Sign in with SSO** button appears on the login page. + +## Behaviour and safety + +**Provisioning and linking.** First sign-in creates a user when auto-provision is on, or +links to an existing local account when `email_verified` matches — no duplicate account. +SSO users have no local password and are labelled with an **SSO** badge under +Users & Roles. + +**Roles are resolved server-side** from your group mapping on every login. Existing users +are never silently demoted when a login carries no matching group, so a temporary IdP +misconfiguration cannot strip your admins. + +**Break-glass.** Local username and password login stays available. The seeded local admin +can always get in, even if the IdP is misconfigured, expired, or unreachable. Do not delete +that account. + +**Behind a reverse proxy.** The redirect URI is derived from the request host. If your +external URL differs from what the backend sees — a proxy, a custom domain, a tunnel — +set **Public base URL** to pin it, or the provider will reject the callback. diff --git a/frontend/package.json b/frontend/package.json index e4089cc..0d46736 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,6 +1,6 @@ { "name": "mikrotik-manager-frontend", - "version": "0.23.3-beta", + "version": "0.23.4-beta", "private": true, "type": "module", "scripts": { diff --git a/frontend/src/version.ts b/frontend/src/version.ts index 856aa47..f597070 100644 --- a/frontend/src/version.ts +++ b/frontend/src/version.ts @@ -1 +1 @@ -export const APP_VERSION = 'v0.23.3 Beta'; +export const APP_VERSION = 'v0.23.4 Beta';