feat: NetFlow/IPFIX traffic analytics — per-client usage, app breakdown, built-in collector (v0.13.0-beta)

Adds Meraki-style traffic analytics powered by MikroTik Traffic Flow:

Backend
- Built-in NetFlow v9 + IPFIX collector (UDP 2055) with hand-rolled decoder,
  template caching, enterprise-field and IPv6 support
- Flow attribution: client IP → MAC via the clients table, upload/download
  direction, NAT-duplicate records dropped
- App classification by protocol/port (~25 categories: HTTPS, QUIC, DNS,
  SSH, Email, WireGuard, ...)
- Deduplication: per-client best-exporter-per-window so flows crossing two
  managed routers are never double-counted
- NAT-tolerant ingest: flows from sources matching no managed device (NAT /
  Tailscale subnet routers in path) are accepted as per-source unidentified
  exporters instead of silently dropped (netflow_accept_unknown setting)
- Storage: InfluxDB client_traffic time-series (bounded cardinality via
  top-N fold) + Postgres client_traffic_daily rollups; daily retention purge
- One-toggle device config push via RouterOS API (/ip traffic-flow + target)
- New /api/traffic/* analytics endpoints + /api/network-services/netflow
- docker-compose: NETFLOW_PORT (default 2055/udp) published on the backend

Frontend
- New Traffic page: fleet up/down chart, top talkers with client
  click-through, app-category donut, 1h–30d ranges
- New NetFlow config page: collector settings, live ingest stats,
  per-device export toggles with health badges, NAT-source banner
- Client detail: App Traffic card (auto-hides without data)
- Clients page: sortable "Data (today)" column
- Settings: NetFlow retention controls

Includes 17 unit tests (decoder fixtures, dedup, attribution, classifier);
verified end-to-end against live RouterOS devices.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2GT-Rich
2026-06-10 10:58:41 -07:00
co-authored by Claude Fable 5
parent 367034c2d6
commit b841eb6e7b
29 changed files with 2553 additions and 15 deletions
+7
View File
@@ -23,6 +23,7 @@ import {
startBulkAddWorker,
stopBulkAddWorker,
} from './services/DeviceBulkAddWorker';
import { netflowCollector } from './services/netflow/NetflowCollector';
import { verifyToken } from './middleware/auth';
import { decrypt } from './utils/crypto';
import { corsMiddlewareOptions, socketIoCorsOptions } from './utils/corsOrigins';
@@ -47,6 +48,7 @@ import configTemplatesRoutes from './routes/configTemplates';
import configHistoryRoutes from './routes/configHistory';
import wirelessRoutes from './routes/wireless';
import networkServicesRoutes from './routes/networkServices';
import trafficAnalyticsRoutes from './routes/trafficAnalytics';
import credentialPresetsRoutes from './routes/credentialPresets';
import systemRoutes from './routes/system';
import { auditMiddleware } from './middleware/auditMiddleware';
@@ -214,6 +216,7 @@ app.use('/api/routers', routersRoutes);
app.use('/api/alerts', alertsRoutes);
app.use('/api/wireless', wirelessRoutes);
app.use('/api/network-services', networkServicesRoutes);
app.use('/api/traffic', trafficAnalyticsRoutes);
app.use('/api/credential-presets', credentialPresetsRoutes);
app.use('/api/audit-log', auditLogRoutes);
app.use('/api/tags', tagsRoutes);
@@ -262,6 +265,9 @@ async function start(): Promise<void> {
await pollerService.start();
await startBulkAddWorker();
// NetFlow/IPFIX collector (binds its UDP socket only when netflow_enabled)
await netflowCollector.start();
// Start HTTP server
httpServer.listen(PORT, '0.0.0.0', () => {
console.log(`✓ Mikrotik Manager backend running on port ${PORT}`);
@@ -271,6 +277,7 @@ async function start(): Promise<void> {
const shutdown = async () => {
console.log('Shutting down...');
await stopBulkAddWorker();
await netflowCollector.stop();
await pollerService.stop();
await redis.quit().catch(() => {});
await pool.end();