feat: NetFlow/IPFIX traffic analytics — per-client usage, app breakdown, built-in collector (v0.13.0-beta)

Adds Meraki-style traffic analytics powered by MikroTik Traffic Flow:

Backend
- Built-in NetFlow v9 + IPFIX collector (UDP 2055) with hand-rolled decoder,
  template caching, enterprise-field and IPv6 support
- Flow attribution: client IP → MAC via the clients table, upload/download
  direction, NAT-duplicate records dropped
- App classification by protocol/port (~25 categories: HTTPS, QUIC, DNS,
  SSH, Email, WireGuard, ...)
- Deduplication: per-client best-exporter-per-window so flows crossing two
  managed routers are never double-counted
- NAT-tolerant ingest: flows from sources matching no managed device (NAT /
  Tailscale subnet routers in path) are accepted as per-source unidentified
  exporters instead of silently dropped (netflow_accept_unknown setting)
- Storage: InfluxDB client_traffic time-series (bounded cardinality via
  top-N fold) + Postgres client_traffic_daily rollups; daily retention purge
- One-toggle device config push via RouterOS API (/ip traffic-flow + target)
- New /api/traffic/* analytics endpoints + /api/network-services/netflow
- docker-compose: NETFLOW_PORT (default 2055/udp) published on the backend

Frontend
- New Traffic page: fleet up/down chart, top talkers with client
  click-through, app-category donut, 1h–30d ranges
- New NetFlow config page: collector settings, live ingest stats,
  per-device export toggles with health badges, NAT-source banner
- Client detail: App Traffic card (auto-hides without data)
- Clients page: sortable "Data (today)" column
- Settings: NetFlow retention controls

Includes 17 unit tests (decoder fixtures, dedup, attribution, classifier);
verified end-to-end against live RouterOS devices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2GT-Rich
2026-06-10 10:58:41 -07:00
co-authored by Claude Fable 5
parent 367034c2d6
commit b841eb6e7b
29 changed files with 2553 additions and 15 deletions
+23
View File
@@ -388,6 +388,19 @@ CREATE TABLE IF NOT EXISTS config_templates (
ALTER TABLE users ADD COLUMN IF NOT EXISTS totp_secret VARCHAR(64);
ALTER TABLE users ADD COLUMN IF NOT EXISTS totp_enabled BOOLEAN NOT NULL DEFAULT false;
-- Per-client daily traffic rollups from the NetFlow collector. mac_address
-- also holds the pseudo-clients 'unknown' (unmapped local IPs) and 'other'
-- (clients folded by the top-N cardinality cap).
CREATE TABLE IF NOT EXISTS client_traffic_daily (
mac_address VARCHAR(17) NOT NULL,
day DATE NOT NULL,
upload_bytes BIGINT NOT NULL DEFAULT 0,
download_bytes BIGINT NOT NULL DEFAULT 0,
app_breakdown JSONB NOT NULL DEFAULT '{}',
PRIMARY KEY (mac_address, day)
);
CREATE INDEX IF NOT EXISTS idx_client_traffic_daily_day ON client_traffic_daily(day DESC);
-- Wireless security profiles (WPA/WPA2/WPA3 config)
CREATE TABLE IF NOT EXISTS wireless_security_profiles (
id SERIAL PRIMARY KEY,
@@ -424,6 +437,16 @@ const DEFAULT_SETTINGS = [
{ key: 'config_snapshot_enabled', value: true },
{ key: 'config_snapshot_interval_min', value: 60 },
{ key: 'config_snapshot_retention', value: 30 },
{ key: 'netflow_enabled', value: false },
{ key: 'netflow_collector_address', value: '' },
{ key: 'netflow_collector_port', value: 2055 },
{ key: 'netflow_version', value: '9' },
{ key: 'netflow_active_timeout', value: '1m' },
{ key: 'netflow_inactive_timeout', value: '15s' },
{ key: 'netflow_topn_clients', value: 50 },
{ key: 'netflow_accept_unknown', value: true },
{ key: 'netflow_retention_days', value: 30 },
{ key: 'netflow_daily_retention_days', value: 365 },
];
export async function runMigrations(): Promise<void> {