feat: NetFlow/IPFIX traffic analytics — per-client usage, app breakdown, built-in collector (v0.13.0-beta)

Adds Meraki-style traffic analytics powered by MikroTik Traffic Flow:

Backend
- Built-in NetFlow v9 + IPFIX collector (UDP 2055) with hand-rolled decoder,
  template caching, enterprise-field and IPv6 support
- Flow attribution: client IP → MAC via the clients table, upload/download
  direction, NAT-duplicate records dropped
- App classification by protocol/port (~25 categories: HTTPS, QUIC, DNS,
  SSH, Email, WireGuard, ...)
- Deduplication: per-client best-exporter-per-window so flows crossing two
  managed routers are never double-counted
- NAT-tolerant ingest: flows from sources matching no managed device (NAT /
  Tailscale subnet routers in path) are accepted as per-source unidentified
  exporters instead of silently dropped (netflow_accept_unknown setting)
- Storage: InfluxDB client_traffic time-series (bounded cardinality via
  top-N fold) + Postgres client_traffic_daily rollups; daily retention purge
- One-toggle device config push via RouterOS API (/ip traffic-flow + target)
- New /api/traffic/* analytics endpoints + /api/network-services/netflow
- docker-compose: NETFLOW_PORT (default 2055/udp) published on the backend

Frontend
- New Traffic page: fleet up/down chart, top talkers with client
  click-through, app-category donut, 1h–30d ranges
- New NetFlow config page: collector settings, live ingest stats,
  per-device export toggles with health badges, NAT-source banner
- Client detail: App Traffic card (auto-hides without data)
- Clients page: sortable "Data (today)" column
- Settings: NetFlow retention controls

Includes 17 unit tests (decoder fixtures, dedup, attribution, classifier);
verified end-to-end against live RouterOS devices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2GT-Rich
2026-06-10 10:58:41 -07:00
co-authored by Claude Fable 5
parent 367034c2d6
commit b841eb6e7b
29 changed files with 2553 additions and 15 deletions
+12 -1
View File
@@ -2,7 +2,7 @@
A self-hosted, full-stack network management platform for MikroTik devices. Monitor, configure, and manage your entire MikroTik infrastructure — routers, switches, and wireless access points — from a single web interface.
![Version](https://img.shields.io/badge/version-0.12.3_Beta-blue)
![Version](https://img.shields.io/badge/version-0.13.0_Beta-blue)
![License](https://img.shields.io/badge/license-AGPLv3-blue)
![Docker](https://img.shields.io/badge/docker-compose-2496ED?logo=docker&logoColor=white)
![TypeScript](https://img.shields.io/badge/TypeScript-5.3-3178C6?logo=typescript&logoColor=white)
@@ -144,6 +144,17 @@ Each service supports multi-device management with conflict detection:
| **NTP** | Server (broadcast/manycast), client (unicast/multicast), sync status |
| **WireGuard** | Interface management, peer configuration, public key display, RX/TX stats |
| **Syslog** | Logging actions (remote/memory/disk/echo) and routing rules; single-device or push-to-all with per-entry coverage; enable/disable rules |
| **NetFlow** | One-toggle Traffic Flow (NetFlow v9/IPFIX) export per device, pointed at the built-in collector; live export status per device |
### Traffic Analytics (NetFlow)
- **Built-in NetFlow/IPFIX collector** — receives Traffic Flow exports from your routers on UDP 2055 (no external tools needed); decoder supports both NetFlow v9 and IPFIX
- **Per-client usage accounting** — flows are attributed to known clients (IP → MAC) so every client shows real upload/download totals; "Data (today)" column on the Clients page and an App Traffic card on each client's detail page
- **Application breakdown** — flows are classified by protocol and port into readable categories (HTTPS, QUIC, DNS, SSH, Email, WireGuard, …) with fleet-wide and per-client views
- **Top talkers** — ranked client bandwidth usage over 1h/24h/7d/30d ranges on the dedicated Traffic page
- **Automatic deduplication** — a flow crossing two managed routers is exported twice; the collector keeps only the best exporter per client each window so totals are never double-counted
- **NAT-tolerant ingest** — when routers export from behind NAT or a VPN/Tailscale subnet router, packets arrive from an address that matches no managed device; the collector accepts these as per-source "unidentified" exporters (configurable) so flows are still attributed to clients instead of silently dropped, and the NetFlow page shows a banner naming the NAT'd source. Per-device export status/flow counters require un-NAT'd sources — exempt `udp/2055` from masquerade on the gateway to restore device identity (and exact cross-device dedup)
- **Fully UI-configurable** — enable the collector, set its address/port/version in the NetFlow page, then toggle export per device; the platform pushes `/ip traffic-flow` configuration to each router via the RouterOS API
- Configurable retention for detailed time-series (default 30 days) and daily rollups (default 365 days)
### Network Topology
- **LLDP-authoritative** — LLDP links are treated as ground truth; CDP/MNDP links to the same neighbor are automatically suppressed, eliminating spurious "Shared Segment" nodes