gustavosantos/MIKROTIK-RMM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(topology,ui): spurious cross-segment links, CPU rounding, serial column (v0.20.1-beta)

Browse Source 
Closes #90, #91, #92.

#90 — Spurious topology links
  Topology treated IP addresses and system identities as unique across the whole
  fleet. They are unique only within a broadcast domain, so an installation with
  physically separate segments that each reuse 10.2.0.0/24 saw devices in one
  segment joined to devices in another.

  The reporter pointed at shared-segment detection, and that is one of four sites
  with the same flaw — but not the first. Step 2 resolves a neighbour IP to a
  managed device from a global first-wins map, so a reused address silently
  resolves to whichever device was loaded first; the union-find in step 6 then
  merges on the resulting device id, a correct-looking key derived from bad data.
  Fixing only step 6 would have left the reported link in place.

  Addresses and identities are now scoped to the device that reported them
  wherever they name a neighbour, so they still collapse one device's duplicate
  sightings but can never join two devices. Resolution and LLDP-coverage
  suppression additionally require the address to be unambiguous fleet-wide.
  MAC addresses and resolved device ids are unaffected — they identify hardware,
  and genuinely shared segments still merge on them.

  Ambiguity detection deliberately ignores MAC divergence. A switch assigns a MAC
  per port, so a device with two uplinks is reported under several MACs; an
  earlier version of this fix treated that as a conflict and refused to resolve
  ordinary multi-port hardware. Caught on a live fleet, where three sightings of
  one CRS510 disagreed on MAC but agreed on identity. The signals used instead are
  an address claimed by two managed devices, or sightings that disagree about the
  neighbour's identity.

  The graph builder is extracted from the route into a pure module with 16 tests,
  including a fixture of the reporter's five-device, three-segment network. It had
  no test coverage before, which is why a flaw this systemic went unnoticed.

  Verified against a live fleet: graph output is identical before and after, and
  the ambiguity list is empty when addressing is clean.

#91 — CPU load not rounded
  The devices table rendered the raw value from a 5-minute Influx mean, so a
  device idling between 1% and 2% displayed as 1.3333333333333333%. Rounded for
  display; the sparkline keeps the unrounded series so its curve stays smooth.

#92 — Serial number in the device table
  Serials were already collected, stored, returned by the API and matched by
  global search, so this is presentation only: a sortable SERIAL column, and
  serial added to the table's filter predicate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2GT-Rich
2026-08-19 16:47:18 -07:00
co-authored by Claude Opus 5
parent 93ff1fbc53
commit b03ea87f11
8 changed files with 676 additions and 294 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1 -1
View File
@@ -2,7 +2,7 @@
A self-hosted, full-stack network management platform for MikroTik devices. Monitor, configure, and manage your entire MikroTik infrastructure — routers, switches, and wireless access points — from a single web interface.
![Version](https://img.shields.io/badge/version-0.20.0_Beta-blue)
![Version](https://img.shields.io/badge/version-0.20.1_Beta-blue)
![License](https://img.shields.io/badge/license-AGPLv3-blue)
![Docker](https://img.shields.io/badge/docker-compose-2496ED?logo=docker&logoColor=white)
![TypeScript](https://img.shields.io/badge/TypeScript-5.3-3178C6?logo=typescript&logoColor=white)
backend/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "mikrotik-manager-backend",
"version": "0.20.0-beta",
"version": "0.20.1-beta",
"description": "Mikrotik Network Management Platform - Backend API",
"main": "dist/index.js",
"scripts": {
backend/src/routes/topology.ts
+14 -287
View File
@@ -2,6 +2,12 @@ import { Router, Request, Response } from 'express';
import { query, queryOne } from '../config/database';
import { requireAuth, requireWrite } from '../middleware/auth';
import { PollerService } from '../services/PollerService';
import {
buildTopology,
type LinkRow,
type ManualLinkRow,
type TopoDevice,
} from '../services/topology/buildTopology';
const router = Router();
router.use(requireAuth);
@@ -11,33 +17,11 @@ export function setPollerService(p: PollerService): void {
pollerService = p;
}
interface LinkRow {
id: number;
from_device_id: number | null;
from_interface: string | null;
to_interface: string | null;
to_device_id: number | null;
neighbor_address: string | null;
neighbor_identity: string | null;
neighbor_platform: string | null;
neighbor_mac: string | null;
stp_role: string | null;
stp_state: string | null;
bridge_name: string | null;
neighbor_caps: string | null;
link_type: string | null;
discovered_by: string | null;
from_device_name: string | null;
to_device_name: string | null;
}
// GET /api/topology
router.get('/', async (_req: Request, res: Response) => {
const PROTO_RANK: Record<string, number> = { lldp: 0, cdp: 1, mndp: 2 };
const protoRank = (p: string | null) => PROTO_RANK[p ?? ''] ?? 3;
const [devices, allLinks, manualLinks] = await Promise.all([
query(
query<TopoDevice>(
`SELECT id, name, ip_address, model, device_type, status, ros_version, ip_addresses_jsonb
FROM devices ORDER BY name ASC`
),
@@ -50,8 +34,7 @@ router.get('/', async (_req: Request, res: Response) => {
LEFT JOIN devices td ON td.id = tl.to_device_id
ORDER BY tl.discovered_at DESC`
),
query<{ id: number; from_device_id: number; to_device_id: number; label: string | null;
from_name: string; to_name: string }>(
query<ManualLinkRow>(
`SELECT ml.*, fd.name AS from_name, td.name AS to_name
FROM manual_topology_links ml
JOIN devices fd ON fd.id = ml.from_device_id
@@ -59,271 +42,15 @@ router.get('/', async (_req: Request, res: Response) => {
),
]);
// ── Step 1: Per-(device, neighbor) best-protocol dedup ─────────────────────
const bestByPair = new Map<string, LinkRow>();
for (const link of allLinks as LinkRow[]) {
if (!link.from_device_id) continue;
const neighborKey = link.to_device_id
? `d:${link.to_device_id}`
: (link.neighbor_mac
? `m:${link.neighbor_mac.toLowerCase()}`
: (link.neighbor_address || link.neighbor_identity || ''));
if (!neighborKey) continue;
const key = `${link.from_device_id}::${neighborKey}`;
const existing = bestByPair.get(key);
if (!existing || protoRank(link.link_type) < protoRank(existing.link_type)) {
bestByPair.set(key, link);
}
}
let links = Array.from(bestByPair.values());
// ── Step 2: Resolve neighbor IP → managed device via all known addresses ───
const stripCidr = (a: string) => { const i = a.indexOf('/'); return i === -1 ? a : a.slice(0, i); };
const normIp = (a: string) => stripCidr(a.trim()).toLowerCase();
const ipToDevice = new Map<string, { id: number; name: string }>();
for (const d of devices as { id: number; name: string; ip_address: string; ip_addresses_jsonb?: unknown }[]) {
if (d.ip_address) {
const k = normIp(d.ip_address);
if (k && !ipToDevice.has(k)) ipToDevice.set(k, { id: d.id, name: d.name });
}
if (Array.isArray(d.ip_addresses_jsonb)) {
for (const entry of d.ip_addresses_jsonb) {
const raw = typeof entry === 'object' && entry && 'address' in entry
? String((entry as { address?: string }).address || '') : '';
const k = normIp(raw);
if (k && !ipToDevice.has(k)) ipToDevice.set(k, { id: d.id, name: d.name });
}
}
}
for (const link of links) {
if (link.to_device_id) continue;
const na = link.neighbor_address?.trim();
if (!na || na.includes('%')) continue;
const hit = ipToDevice.get(normIp(na));
if (hit && hit.id !== link.from_device_id) {
link.to_device_id = hit.id;
link.to_device_name = hit.name;
}
}
// ── Step 3: Build "LLDP-covered" neighbor set ───────────────────────────────
// Any neighbor identifier (device_id / MAC / IP) that appears as the target of
// an LLDP link. We use this to suppress CDP/MNDP links to the same neighbor —
// LLDP already gives us a precise point-to-point path, no need for the noisy L2 data.
const lldpCoveredDeviceIds = new Set<number>();
const lldpCoveredMacs = new Set<string>();
const lldpCoveredAddresses = new Set<string>();
for (const link of links) {
if (link.link_type !== 'lldp') continue;
if (link.to_device_id) lldpCoveredDeviceIds.add(link.to_device_id);
if (link.neighbor_mac) lldpCoveredMacs.add(link.neighbor_mac.toLowerCase());
if (link.neighbor_address) lldpCoveredAddresses.add(normIp(link.neighbor_address));
}
const isLldpCovered = (link: LinkRow): boolean => {
if (link.to_device_id && lldpCoveredDeviceIds.has(link.to_device_id)) return true;
if (link.neighbor_mac && lldpCoveredMacs.has(link.neighbor_mac.toLowerCase())) return true;
if (link.neighbor_address && lldpCoveredAddresses.has(normIp(link.neighbor_address))) return true;
return false;
};
// Drop non-LLDP links whose neighbor is already reachable via LLDP elsewhere
links = links.filter((l) => l.link_type === 'lldp' || !isLldpCovered(l));
// ── Step 4: Merge bidirectional LLDP pairs ─────────────────────────────────
// For LLDP links A→B and B→A between two managed devices, keep one canonical
// record (lower from_device_id is "from"). Enrich to_interface from the
// reverse record if the forward record is missing it.
const canonicalLldp: LinkRow[] = [];
const mergedPairs = new Set<string>();
for (const link of links) {
if (link.link_type !== 'lldp' || !link.to_device_id) {
canonicalLldp.push(link);
continue;
}
const lo = Math.min(link.from_device_id!, link.to_device_id);
const hi = Math.max(link.from_device_id!, link.to_device_id);
const pairKey = `${lo}:${hi}`;
if (mergedPairs.has(pairKey)) continue;
mergedPairs.add(pairKey);
// Prefer the direction where from_device_id is the lower id for stability
const forward = link.from_device_id === lo ? link : links.find(
r => r.link_type === 'lldp' && r.from_device_id === lo && r.to_device_id === hi
);
const reverse = links.find(
r => r.link_type === 'lldp' && r.from_device_id === hi && r.to_device_id === lo
);
const canonical = forward ?? link;
// If to_interface is missing on the canonical record but the reverse has from_interface, fill it in
if (!canonical.to_interface && reverse?.from_interface) {
canonical.to_interface = reverse.from_interface;
}
// If from_interface is missing but reverse has to_interface, fill it in
if (!canonical.from_interface && reverse?.to_interface) {
canonical.from_interface = reverse.to_interface;
}
canonicalLldp.push(canonical);
}
links = canonicalLldp;
// ── Step 5: Build external node map ────────────────────────────────────────
const externalMap = new Map<string, {
id: string; name: string; address: string; platform: string; mac: string; caps: string;
}>();
for (const link of links) {
if (link.to_device_id) continue;
const key = link.neighbor_mac || link.neighbor_address || link.neighbor_identity || '';
if (!key) continue;
if (!externalMap.has(key)) {
const safeId = key.replace(/[.: ]/g, '');
externalMap.set(key, {
id: `ext-${safeId}`,
name: link.neighbor_identity || link.neighbor_address || 'Unknown',
address: link.neighbor_address || '',
platform: link.neighbor_platform || '',
mac: link.neighbor_mac || '',
caps: link.neighbor_caps || '',
});
}
}
// ── Step 6: Shared-segment detection (only for truly unresolved non-LLDP links) ──
// After LLDP suppression above, the remaining non-LLDP links are genuinely
// on segments where we couldn't find LLDP. Only group them into shared segments
// when a single port sees ≥2 distinct unresolved neighbors.
const lldpLinks = links.filter((l) => l.link_type === 'lldp');
const nonLldpLinks = links.filter((l) => l.link_type !== 'lldp' && !!l.from_device_id);
const portGroupMap = new Map<string, LinkRow[]>();
for (const link of nonLldpLinks) {
const pk = `${link.from_device_id}::${link.from_interface ?? ''}`;
if (!portGroupMap.has(pk)) portGroupMap.set(pk, []);
portGroupMap.get(pk)!.push(link);
}
const sharedPortKeys = [...portGroupMap.keys()].filter((pk) => portGroupMap.get(pk)!.length >= 2);
const soloNonLldp = [...portGroupMap.values()].filter((g) => g.length < 2).flat();
const nKey = (l: LinkRow) =>
l.to_device_id ? `d:${l.to_device_id}` :
l.neighbor_mac ? `m:${l.neighbor_mac.toLowerCase()}` :
l.neighbor_address ? `a:${l.neighbor_address}` :
`i:${l.neighbor_identity ?? ''}`;
// Union-find to merge port groups sharing a common neighbor
const ufParent = new Map<string, string>(sharedPortKeys.map((k) => [k, k]));
const ufFind = (k: string): string => {
if (ufParent.get(k) !== k) ufParent.set(k, ufFind(ufParent.get(k)!));
return ufParent.get(k)!;
};
const ufUnion = (a: string, b: string) => ufParent.set(ufFind(a), ufFind(b));
const pkNeighborSets = new Map<string, Set<string>>();
for (const pk of sharedPortKeys) {
pkNeighborSets.set(pk, new Set(portGroupMap.get(pk)!.map(nKey)));
}
for (let i = 0; i < sharedPortKeys.length; i++) {
for (let j = i + 1; j < sharedPortKeys.length; j++) {
const setA = pkNeighborSets.get(sharedPortKeys[i])!;
for (const n of pkNeighborSets.get(sharedPortKeys[j])!) {
if (setA.has(n)) { ufUnion(sharedPortKeys[i], sharedPortKeys[j]); break; }
}
}
}
const segGroups = new Map<string, string[]>();
for (const pk of sharedPortKeys) {
const root = ufFind(pk);
if (!segGroups.has(root)) segGroups.set(root, []);
segGroups.get(root)!.push(pk);
}
interface SegConn { src: string; dst: string; port: string; }
const segNodes: { id: string; name: string; address: string; platform: string; mac: string; caps: string }[] = [];
const segConns: SegConn[] = [];
for (const [root, pks] of segGroups) {
const segId = `seg-${root.replace(/[^a-z0-9]/gi, '')}`;
const srcDevPorts = new Map<string, string>();
const allDevIds = new Set<string>();
const extNKeys = new Set<string>();
for (const pk of pks) {
const colonIdx = pk.indexOf('::');
const devId = pk.slice(0, colonIdx);
const port = pk.slice(colonIdx + 2);
srcDevPorts.set(devId, port);
allDevIds.add(devId);
for (const link of portGroupMap.get(pk)!) {
if (link.to_device_id) allDevIds.add(String(link.to_device_id));
else extNKeys.add(nKey(link));
}
}
segNodes.push({
id: segId,
name: 'Shared Segment',
address: '',
platform: `${allDevIds.size} devices`,
mac: '',
caps: 'segment',
});
for (const [devId, port] of srcDevPorts) {
segConns.push({ src: devId, dst: segId, port });
}
for (const nk of extNKeys) {
const ext = Array.from(externalMap.values()).find((e) =>
`m:${(e.mac || '').toLowerCase()}` === nk ||
`a:${e.address}` === nk ||
`i:${e.name}` === nk
);
if (ext) segConns.push({ src: ext.id, dst: segId, port: '' });
}
}
const activeLinks = [...lldpLinks, ...soloNonLldp];
const externalNodes = Array.from(externalMap.values());
const allExtNodes = [...externalNodes, ...segNodes];
// ── Step 7: Include manual links as synthetic topology link rows ────────────
const manualAsLinks: LinkRow[] = (manualLinks as { id: number; from_device_id: number; to_device_id: number; label: string | null; from_name: string; to_name: string }[]).map((ml) => ({
id: -ml.id, // negative id to avoid collision
from_device_id: ml.from_device_id,
from_interface: ml.label ?? null,
to_interface: null,
to_device_id: ml.to_device_id,
neighbor_address: null,
neighbor_identity: null,
neighbor_platform: null,
neighbor_mac: null,
stp_role: null,
stp_state: null,
bridge_name: null,
neighbor_caps: null,
link_type: 'manual',
discovered_by: null,
from_device_name: ml.from_name,
to_device_name: ml.to_name,
}));
const graph = buildTopology(devices, allLinks, manualLinks);
res.json({
devices,
links: [...activeLinks, ...manualAsLinks],
externalNodes: allExtNodes,
segConns,
manualLinkIds: (manualLinks as { id: number; from_device_id: number; to_device_id: number }[]).map((ml) => ({
links: graph.links,
externalNodes: graph.externalNodes,
segConns: graph.segConns,
ambiguous: graph.ambiguous,
manualLinkIds: manualLinks.map((ml) => ({
id: ml.id,
from_device_id: ml.from_device_id,
to_device_id: ml.to_device_id,
backend/src/services/topology/__tests__/buildTopology.test.ts
+211
View File
@@ -0,0 +1,211 @@
import { buildTopology, findAmbiguousIdentifiers, type LinkRow, type TopoDevice } from '../buildTopology';
let nextId = 1;
function link(over: Partial<LinkRow>): LinkRow {
return {
id: nextId++,
from_device_id: null, from_interface: null, to_interface: null, to_device_id: null,
neighbor_address: null, neighbor_identity: null, neighbor_platform: null, neighbor_mac: null,
stp_role: null, stp_state: null, bridge_name: null, neighbor_caps: null,
link_type: 'mndp', discovered_by: null, from_device_name: null, to_device_name: null,
...over,
};
}
const dev = (id: number, name: string, ip: string, extra: string[] = []): TopoDevice => ({
id, name, ip_address: ip,
ip_addresses_jsonb: extra.map((a) => ({ address: a })),
});
beforeEach(() => { nextId = 1; });
/**
* The network from issue #90: three physically disconnected segments, each with a
* VLAN named `management_vlan`, each reusing 10.2.0.0/24. gamma (segment 2) and
* delta (segment 3) both answer on 10.2.0.1. Nothing in segment 1 can reach
* segment 3, so no link between alpha and delta may ever appear.
*/
function segmentedFleet() {
const devices = [
dev(1, 'alpha', '192.168.1.10', ['10.2.0.1']),
dev(2, 'beta', '192.168.1.11', ['10.2.0.2']),
dev(3, 'gamma', '192.168.1.12', ['10.2.0.1']),
dev(4, 'delta', '192.168.1.13', ['10.2.0.1']),
dev(5, 'lambda', '192.168.1.14', ['10.2.0.2']),
];
return devices;
}
describe('findAmbiguousIdentifiers', () => {
it('flags an address claimed by more than one managed device', () => {
const amb = findAmbiguousIdentifiers(segmentedFleet(), []);
expect(amb.addresses.has('10.2.0.1')).toBe(true); // alpha, gamma, delta
expect(amb.addresses.has('10.2.0.2')).toBe(true); // beta, lambda
expect(amb.addresses.has('192.168.1.10')).toBe(false);
});
it('flags an address whose sightings disagree about who answers to it', () => {
const amb = findAmbiguousIdentifiers([], [
link({ from_device_id: 1, neighbor_address: '10.9.9.1', neighbor_identity: 'gamma' }),
link({ from_device_id: 4, neighbor_address: '10.9.9.1', neighbor_identity: 'delta' }),
]);
expect(amb.addresses.has('10.9.9.1')).toBe(true);
});
/**
* Real data from a CRS510: a switch assigns a MAC per port, so the same device is
* reported as ...74:0E by two neighbours and ...74:0F by a third. All three agree
* on the identity, so this is one device with several uplinks — not a conflict.
* An earlier version treated MAC divergence as ambiguity and refused to resolve
* ordinary multi-port hardware.
*/
it('does not flag a multi-port device reported under several port MACs', () => {
const amb = findAmbiguousIdentifiers([dev(1, '2GT-NW-100G', '192.168.0.40')], [
link({ from_device_id: 8, neighbor_address: '192.168.0.40', neighbor_mac: 'F4:1E:57:51:74:0E', neighbor_identity: '2GT-NW-100G', link_type: 'cdp' }),
link({ from_device_id: 7, neighbor_address: '192.168.0.40', neighbor_mac: 'F4:1E:57:51:74:0F', neighbor_identity: '2GT-NW-100G', link_type: 'lldp' }),
link({ from_device_id: 2, neighbor_address: '192.168.0.40', neighbor_mac: 'F4:1E:57:51:74:0E', neighbor_identity: '2GT-NW-100G', link_type: 'cdp' }),
]);
expect(amb.addresses.has('192.168.0.40')).toBe(false);
});
it('flags an identity claimed by two managed devices', () => {
const amb = findAmbiguousIdentifiers(
[dev(1, 'MikroTik', '10.0.0.1'), dev(2, 'MikroTik', '10.0.0.2')],
[]
);
expect(amb.identities.has('mikrotik')).toBe(true);
});
});
describe('buildTopology — disconnected segments reusing addresses (#90)', () => {
it('does not resolve a neighbour to a device in another segment', () => {
// alpha sees its own segment's 10.2.0.1 over MNDP with no MAC available.
const graph = buildTopology(segmentedFleet(), [
link({ from_device_id: 1, from_interface: 'ether1', neighbor_address: '10.2.0.1' }),
], []);
const resolved = graph.links.filter((l) => l.to_device_id != null);
expect(resolved).toHaveLength(0); // never silently becomes gamma or delta
});
it('does not join two segments into one shared segment', () => {
const links = [
// alpha's trunk port sees two unresolved neighbours in segment 1
link({ from_device_id: 1, from_interface: 'ether1', neighbor_address: '10.2.0.1' }),
link({ from_device_id: 1, from_interface: 'ether1', neighbor_address: '10.2.0.2' }),
// delta's trunk port sees two unresolved neighbours in segment 3,
// which happen to carry the very same addresses
link({ from_device_id: 4, from_interface: 'ether1', neighbor_address: '10.2.0.1' }),
link({ from_device_id: 4, from_interface: 'ether1', neighbor_address: '10.2.0.2' }),
];
const graph = buildTopology(segmentedFleet(), links, []);
const segments = graph.externalNodes.filter((n) => n.caps === 'segment');
expect(segments).toHaveLength(2); // one per port group, NOT merged into one
// No segment may be attached to both alpha and delta.
for (const seg of segments) {
const attached = graph.segConns.filter((c) => c.dst === seg.id).map((c) => c.src);
expect(attached).not.toEqual(expect.arrayContaining(['1', '4']));
}
});
it('keeps same-addressed neighbours in different segments as separate nodes', () => {
const graph = buildTopology(segmentedFleet(), [
link({ from_device_id: 1, from_interface: 'ether1', neighbor_address: '10.2.0.9', neighbor_identity: 'switch' }),
link({ from_device_id: 4, from_interface: 'ether1', neighbor_address: '10.2.0.9', neighbor_identity: 'switch' }),
], []);
// Both report the same address with no MAC; the fleet proves 10.2.0.x is reused,
// so they must not collapse into a single external node bridging the segments.
const ext = graph.externalNodes.filter((n) => n.caps !== 'segment');
expect(ext).toHaveLength(2);
});
it('still merges genuinely identical neighbours when a MAC proves identity', () => {
const links = [
link({ from_device_id: 1, from_interface: 'ether1', neighbor_mac: 'AA:BB:CC:00:00:01', neighbor_address: '10.2.0.1' }),
link({ from_device_id: 1, from_interface: 'ether1', neighbor_mac: 'AA:BB:CC:00:00:02', neighbor_address: '10.2.0.2' }),
link({ from_device_id: 2, from_interface: 'ether5', neighbor_mac: 'AA:BB:CC:00:00:01', neighbor_address: '10.2.0.1' }),
link({ from_device_id: 2, from_interface: 'ether5', neighbor_mac: 'AA:BB:CC:00:00:02', neighbor_address: '10.2.0.2' }),
];
const graph = buildTopology(segmentedFleet(), links, []);
const segments = graph.externalNodes.filter((n) => n.caps === 'segment');
// Same MACs on both ports: this really is one shared segment.
expect(segments).toHaveLength(1);
const attached = graph.segConns.filter((c) => c.dst === segments[0].id).map((c) => c.src).sort();
expect(attached).toEqual(expect.arrayContaining(['1', '2']));
});
it('reports which identifiers it distrusted', () => {
const graph = buildTopology(segmentedFleet(), [], []);
expect(graph.ambiguous.addresses.sort()).toEqual(['10.2.0.1', '10.2.0.2']);
});
});
describe('buildTopology — behaviour preserved for unambiguous fleets', () => {
const flat = [dev(1, 'sw1', '192.168.0.1'), dev(2, 'sw2', '192.168.0.2')];
it('resolves a neighbour IP to a managed device', () => {
const graph = buildTopology(flat, [
link({ from_device_id: 1, from_interface: 'ether1', neighbor_address: '192.168.0.2' }),
], []);
expect(graph.links[0].to_device_id).toBe(2);
expect(graph.links[0].to_device_name).toBe('sw2');
});
it('prefers LLDP over MNDP for the same neighbour', () => {
const graph = buildTopology(flat, [
link({ from_device_id: 1, from_interface: 'e1', to_device_id: 2, link_type: 'mndp' }),
link({ from_device_id: 1, from_interface: 'e1', to_device_id: 2, link_type: 'lldp' }),
], []);
expect(graph.links).toHaveLength(1);
expect(graph.links[0].link_type).toBe('lldp');
});
it('suppresses a CDP link to a neighbour LLDP already describes', () => {
// Coverage is per-neighbour: sw2 is described precisely by LLDP from sw1, so
// sw3's noisier CDP sighting of sw2 adds nothing and is dropped.
const three = [...flat, dev(3, 'sw3', '192.168.0.3')];
const graph = buildTopology(three, [
link({ from_device_id: 1, from_interface: 'e1', to_device_id: 2, link_type: 'lldp' }),
link({ from_device_id: 3, from_interface: 'e2', to_device_id: 2, link_type: 'cdp' }),
], []);
expect(graph.links).toHaveLength(1);
expect(graph.links[0].link_type).toBe('lldp');
});
it('keeps a CDP link when LLDP does not describe that neighbour at all', () => {
// Asymmetric discovery — the case the reporting installation is in. sw2 has no
// LLDP view of sw1, so dropping its CDP link would erase a real adjacency.
const graph = buildTopology(flat, [
link({ from_device_id: 1, from_interface: 'e1', to_device_id: 2, link_type: 'lldp' }),
link({ from_device_id: 2, from_interface: 'e9', to_device_id: 1, link_type: 'cdp' }),
], []);
expect(graph.links).toHaveLength(2);
});
it('merges a bidirectional LLDP pair into one canonical link', () => {
const graph = buildTopology(flat, [
link({ from_device_id: 2, from_interface: 'e9', to_device_id: 1, link_type: 'lldp' }),
link({ from_device_id: 1, from_interface: 'e1', to_device_id: 2, link_type: 'lldp' }),
], []);
expect(graph.links).toHaveLength(1);
expect(graph.links[0].from_device_id).toBe(1);
expect(graph.links[0].to_interface).toBe('e9');
});
it('passes manual links through as synthetic rows', () => {
const graph = buildTopology(flat, [], [
{ id: 7, from_device_id: 1, to_device_id: 2, label: 'fibre', from_name: 'sw1', to_name: 'sw2' },
]);
const manual = graph.links.find((l) => l.link_type === 'manual');
expect(manual?.id).toBe(-7);
expect(manual?.from_interface).toBe('fibre');
});
it('does not link a device to itself through its own address', () => {
const graph = buildTopology(flat, [
link({ from_device_id: 1, from_interface: 'e1', neighbor_address: '192.168.0.1' }),
], []);
expect(graph.links[0].to_device_id).toBeNull();
});
});
backend/src/services/topology/buildTopology.ts
+428
View File
@@ -0,0 +1,428 @@
/**
* Topology graph construction.
*
* Extracted from the route handler so the graph rules can be exercised against
* fixtures — the logic is subtle, order-dependent, and its failure mode is a
* plausible-looking map rather than an error.
*
* ## Identifier uniqueness, and why it matters
*
* Neighbour discovery gives us four ways to name the far end of a link: a resolved
* managed device id, a MAC address, an IP address, and a system identity. Only the
* first two are unique across a fleet. **IP addresses and identities are unique only
* within a broadcast domain.**
*
* Treating them as globally unique fabricates links between physically separate
* networks: an installation with several disconnected segments that each reuse
* `10.2.0.0/24` and each name their management VLAN the same thing will see devices
* in segment A joined to devices in segment C, because both segments contain a
* `10.2.0.1` (github.com/2GT-Media-Group-LLC/mikrotik-manager/issues/90).
*
* The defence is to detect identifiers that demonstrably are not unique and refuse
* to merge on them, rather than to invent a notion of "segment" the data cannot
* support (the reporting installation uses the same VLAN name everywhere, so even
* the VLAN is not a discriminator). An ambiguous identifier is scoped to the device
* that reported it, so it still deduplicates that device's own view while never
* joining two devices together.
*/
export interface TopoDevice {
id: number;
name: string;
ip_address: string;
ip_addresses_jsonb?: unknown;
[key: string]: unknown;
}
export interface LinkRow {
id: number;
from_device_id: number | null;
from_interface: string | null;
to_interface: string | null;
to_device_id: number | null;
neighbor_address: string | null;
neighbor_identity: string | null;
neighbor_platform: string | null;
neighbor_mac: string | null;
stp_role: string | null;
stp_state: string | null;
bridge_name: string | null;
neighbor_caps: string | null;
link_type: string | null;
discovered_by: string | null;
from_device_name: string | null;
to_device_name: string | null;
}
export interface ManualLinkRow {
id: number;
from_device_id: number;
to_device_id: number;
label: string | null;
from_name: string;
to_name: string;
}
export interface ExternalNode {
id: string;
name: string;
address: string;
platform: string;
mac: string;
caps: string;
}
export interface SegConn { src: string; dst: string; port: string; }
export interface TopologyGraph {
links: LinkRow[];
externalNodes: ExternalNode[];
segConns: SegConn[];
/**
* Identifiers found on more than one device or neighbour. Surfaced so the cause
* of a sparser-than-expected map is inspectable rather than mysterious.
*/
ambiguous: { addresses: string[]; identities: string[] };
}
const PROTO_RANK: Record<string, number> = { lldp: 0, cdp: 1, mndp: 2 };
const protoRank = (p: string | null) => PROTO_RANK[p ?? ''] ?? 3;
const stripCidr = (a: string) => { const i = a.indexOf('/'); return i === -1 ? a : a.slice(0, i); };
const normIp = (a: string) => stripCidr(a.trim()).toLowerCase();
const normId = (s: string) => s.trim().toLowerCase();
/** Every address a managed device answers on, normalised. */
function deviceAddresses(d: TopoDevice): string[] {
const out: string[] = [];
if (d.ip_address) out.push(normIp(d.ip_address));
if (Array.isArray(d.ip_addresses_jsonb)) {
for (const entry of d.ip_addresses_jsonb) {
const raw = typeof entry === 'object' && entry && 'address' in entry
? String((entry as { address?: string }).address || '') : '';
const k = normIp(raw);
if (k) out.push(k);
}
}
return out.filter(Boolean);
}
/**
* Identify addresses and identities that cannot be trusted to name one thing.
*
* This drives two decisions only: whether an address may resolve to a managed
* device (Step 2), and whether it may mark a neighbour as already covered by LLDP
* (Step 3). Both ask the same question — does this address identify exactly one
* device across the whole fleet?
*
* Two signals answer it:
* - more than one managed device claims the address
* - sightings of the address disagree about the neighbour's identity, which means
* at least two different devices answer to it
*
* Deliberately NOT a signal: seeing the address with more than one MAC. A switch
* assigns a MAC per port, so a device with two uplinks is legitimately reported as
* `F4:1E:57:51:74:0E` from one neighbour and `…:0F` from another. Treating that as a
* conflict would refuse to resolve ordinary multi-port hardware — observed on a real
* fleet, where all three sightings agreed on the identity `2GT-NW-100G`.
*/
export function findAmbiguousIdentifiers(
devices: TopoDevice[],
links: LinkRow[]
): { addresses: Set<string>; identities: Set<string> } {
const addrDevices = new Map<string, Set<number>>();
const addrIdentities = new Map<string, Set<string>>();
const identDevices = new Map<string, Set<number>>();
const add = <K, V>(m: Map<K, Set<V>>, k: K, v: V) => {
const s = m.get(k) ?? new Set<V>();
s.add(v);
m.set(k, s);
};
for (const d of devices) {
for (const a of deviceAddresses(d)) add(addrDevices, a, d.id);
if (d.name) add(identDevices, normId(d.name), d.id);
}
for (const l of links) {
if (!l.neighbor_address || !l.neighbor_identity) continue;
add(addrIdentities, normIp(l.neighbor_address), normId(l.neighbor_identity));
}
const addresses = new Set<string>();
for (const [a, s] of addrDevices) if (s.size > 1) addresses.add(a);
for (const [a, s] of addrIdentities) if (s.size > 1) addresses.add(a);
const identities = new Set<string>();
for (const [i, s] of identDevices) if (s.size > 1) identities.add(i);
return { addresses, identities };
}
/**
* A stable name for the far end of a link.
*
* A resolved device id or a MAC identifies hardware, and hardware is unique — those
* keys are global. An address or a bare identity is not proof of identity at all, so
* those keys are always scoped to the device that reported them. Two devices that
* each see "10.2.0.1" therefore describe two neighbours unless a MAC says otherwise.
*
* This costs nothing in practice: RouterOS neighbour discovery reports a MAC for
* MNDP and CDP alike, so genuinely shared segments still merge on the MAC. Where a
* MAC really is missing we degrade to one node per reporting device — visibly
* duplicated, which an operator can reason about, rather than a fabricated adjacency
* between networks that cannot reach each other.
*/
function makeNeighborKey() {
return (l: LinkRow): string | null => {
if (l.to_device_id) return `d:${l.to_device_id}`;
if (l.neighbor_mac) return `m:${l.neighbor_mac.toLowerCase()}`;
const scope = `@${l.from_device_id ?? '?'}`;
if (l.neighbor_address) return `a:${normIp(l.neighbor_address)}${scope}`;
if (l.neighbor_identity) return `i:${normId(l.neighbor_identity)}${scope}`;
return null;
};
}
export function buildTopology(
devices: TopoDevice[],
allLinks: LinkRow[],
manualLinks: ManualLinkRow[]
): TopologyGraph {
const amb = findAmbiguousIdentifiers(devices, allLinks);
const neighborKey = makeNeighborKey();
// ── Step 1: Per-(device, neighbor) best-protocol dedup ─────────────────────
const bestByPair = new Map<string, LinkRow>();
for (const link of allLinks) {
if (!link.from_device_id) continue;
const nk = neighborKey(link);
if (!nk) continue;
const key = `${link.from_device_id}::${nk}`;
const existing = bestByPair.get(key);
if (!existing || protoRank(link.link_type) < protoRank(existing.link_type)) {
bestByPair.set(key, link);
}
}
let links = Array.from(bestByPair.values());
// ── Step 2: Resolve neighbor IP → managed device ───────────────────────────
// Only unambiguous addresses may resolve. An address living on two devices tells
// us nothing about which one a neighbour actually is, and guessing produces a
// link between networks that cannot reach each other.
const ipToDevice = new Map<string, { id: number; name: string }>();
for (const d of devices) {
for (const a of deviceAddresses(d)) {
if (amb.addresses.has(a)) continue;
if (!ipToDevice.has(a)) ipToDevice.set(a, { id: d.id, name: d.name });
}
}
for (const link of links) {
if (link.to_device_id) continue;
const na = link.neighbor_address?.trim();
if (!na || na.includes('%')) continue;
const hit = ipToDevice.get(normIp(na));
if (hit && hit.id !== link.from_device_id) {
link.to_device_id = hit.id;
link.to_device_name = hit.name;
}
}
// ── Step 3: Build "LLDP-covered" neighbor set ───────────────────────────────
// Suppress CDP/MNDP links to a neighbour LLDP already describes precisely. Only
// globally unique identifiers may suppress: an ambiguous address would silence a
// real link in another segment that merely reuses the same numbering.
const lldpCoveredDeviceIds = new Set<number>();
const lldpCoveredMacs = new Set<string>();
const lldpCoveredAddresses = new Set<string>();
for (const link of links) {
if (link.link_type !== 'lldp') continue;
if (link.to_device_id) lldpCoveredDeviceIds.add(link.to_device_id);
if (link.neighbor_mac) lldpCoveredMacs.add(link.neighbor_mac.toLowerCase());
if (link.neighbor_address) {
const a = normIp(link.neighbor_address);
if (!amb.addresses.has(a)) lldpCoveredAddresses.add(a);
}
}
const isLldpCovered = (link: LinkRow): boolean => {
if (link.to_device_id && lldpCoveredDeviceIds.has(link.to_device_id)) return true;
if (link.neighbor_mac && lldpCoveredMacs.has(link.neighbor_mac.toLowerCase())) return true;
if (link.neighbor_address && lldpCoveredAddresses.has(normIp(link.neighbor_address))) return true;
return false;
};
links = links.filter((l) => l.link_type === 'lldp' || !isLldpCovered(l));
// ── Step 4: Merge bidirectional LLDP pairs ─────────────────────────────────
const canonicalLldp: LinkRow[] = [];
const mergedPairs = new Set<string>();
for (const link of links) {
if (link.link_type !== 'lldp' || !link.to_device_id) {
canonicalLldp.push(link);
continue;
}
const lo = Math.min(link.from_device_id!, link.to_device_id);
const hi = Math.max(link.from_device_id!, link.to_device_id);
const pairKey = `${lo}:${hi}`;
if (mergedPairs.has(pairKey)) continue;
mergedPairs.add(pairKey);
const forward = link.from_device_id === lo ? link : links.find(
(r) => r.link_type === 'lldp' && r.from_device_id === lo && r.to_device_id === hi
);
const reverse = links.find(
(r) => r.link_type === 'lldp' && r.from_device_id === hi && r.to_device_id === lo
);
const canonical = forward ?? link;
if (!canonical.to_interface && reverse?.from_interface) canonical.to_interface = reverse.from_interface;
if (!canonical.from_interface && reverse?.to_interface) canonical.from_interface = reverse.to_interface;
canonicalLldp.push(canonical);
}
links = canonicalLldp;
// ── Step 5: Build external node map ────────────────────────────────────────
// Keyed by the same neighbour key used everywhere else, so two unmanaged devices
// that merely share an address in disconnected segments stay two nodes.
const externalMap = new Map<string, ExternalNode>();
for (const link of links) {
if (link.to_device_id) continue;
const key = neighborKey(link);
if (!key) continue;
if (!externalMap.has(key)) {
externalMap.set(key, {
id: `ext-${key.replace(/[^a-z0-9]/gi, '')}`,
name: link.neighbor_identity || link.neighbor_address || 'Unknown',
address: link.neighbor_address || '',
platform: link.neighbor_platform || '',
mac: link.neighbor_mac || '',
caps: link.neighbor_caps || '',
});
}
}
// ── Step 6: Shared-segment detection ───────────────────────────────────────
const lldpLinks = links.filter((l) => l.link_type === 'lldp');
const nonLldpLinks = links.filter((l) => l.link_type !== 'lldp' && !!l.from_device_id);
const portGroupMap = new Map<string, LinkRow[]>();
for (const link of nonLldpLinks) {
const pk = `${link.from_device_id}::${link.from_interface ?? ''}`;
if (!portGroupMap.has(pk)) portGroupMap.set(pk, []);
portGroupMap.get(pk)!.push(link);
}
const sharedPortKeys = [...portGroupMap.keys()].filter((pk) => portGroupMap.get(pk)!.length >= 2);
const soloNonLldp = [...portGroupMap.values()].filter((g) => g.length < 2).flat();
// Union-find to merge port groups sharing a common neighbour. Merging asserts two
// ports face the same physical segment, so it may only rest on an identifier that
// is unique fleet-wide — a device-scoped key can never match another device's.
const ufParent = new Map<string, string>(sharedPortKeys.map((k) => [k, k]));
const ufFind = (k: string): string => {
if (ufParent.get(k) !== k) ufParent.set(k, ufFind(ufParent.get(k)!));
return ufParent.get(k)!;
};
const ufUnion = (a: string, b: string) => ufParent.set(ufFind(a), ufFind(b));
const pkNeighborSets = new Map<string, Set<string>>();
for (const pk of sharedPortKeys) {
pkNeighborSets.set(
pk,
new Set(portGroupMap.get(pk)!.map(neighborKey).filter((k): k is string => k !== null))
);
}
for (let i = 0; i < sharedPortKeys.length; i++) {
for (let j = i + 1; j < sharedPortKeys.length; j++) {
const setA = pkNeighborSets.get(sharedPortKeys[i])!;
for (const n of pkNeighborSets.get(sharedPortKeys[j])!) {
if (setA.has(n)) { ufUnion(sharedPortKeys[i], sharedPortKeys[j]); break; }
}
}
}
const segGroups = new Map<string, string[]>();
for (const pk of sharedPortKeys) {
const root = ufFind(pk);
if (!segGroups.has(root)) segGroups.set(root, []);
segGroups.get(root)!.push(pk);
}
const segNodes: ExternalNode[] = [];
const segConns: SegConn[] = [];
for (const [root, pks] of segGroups) {
const segId = `seg-${root.replace(/[^a-z0-9]/gi, '')}`;
const srcDevPorts = new Map<string, string>();
const allDevIds = new Set<string>();
const extKeys = new Set<string>();
for (const pk of pks) {
const colonIdx = pk.indexOf('::');
const devId = pk.slice(0, colonIdx);
const port = pk.slice(colonIdx + 2);
srcDevPorts.set(devId, port);
allDevIds.add(devId);
for (const link of portGroupMap.get(pk)!) {
if (link.to_device_id) allDevIds.add(String(link.to_device_id));
else {
const k = neighborKey(link);
if (k) extKeys.add(k);
}
}
}
segNodes.push({
id: segId,
name: 'Shared Segment',
address: '',
platform: `${allDevIds.size} devices`,
mac: '',
caps: 'segment',
});
for (const [devId, port] of srcDevPorts) segConns.push({ src: devId, dst: segId, port });
// Direct key lookup — Step 5 keys the map by the same function, so no
// reconstruct-and-compare guesswork.
for (const k of extKeys) {
const ext = externalMap.get(k);
if (ext) segConns.push({ src: ext.id, dst: segId, port: '' });
}
}
// ── Step 7: Manual links as synthetic rows ─────────────────────────────────
const manualAsLinks: LinkRow[] = manualLinks.map((ml) => ({
id: -ml.id,
from_device_id: ml.from_device_id,
from_interface: ml.label ?? null,
to_interface: null,
to_device_id: ml.to_device_id,
neighbor_address: null,
neighbor_identity: null,
neighbor_platform: null,
neighbor_mac: null,
stp_role: null,
stp_state: null,
bridge_name: null,
neighbor_caps: null,
link_type: 'manual',
discovered_by: null,
from_device_name: ml.from_name,
to_device_name: ml.to_name,
}));
return {
links: [...lldpLinks, ...soloNonLldp, ...manualAsLinks],
externalNodes: [...externalMap.values(), ...segNodes],
segConns,
ambiguous: { addresses: [...amb.addresses], identities: [...amb.identities] },
};
}
frontend/package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "mikrotik-manager-frontend",
"version": "0.20.0-beta",
"version": "0.20.1-beta",
"private": true,
"type": "module",
"scripts": {
frontend/src/pages/DevicesPage.tsx
+19 -3
View File
@@ -14,7 +14,7 @@ import AddDeviceModal from '../components/devices/AddDeviceModal';
import EditDeviceModal from '../components/devices/EditDeviceModal';
import TryAllDiscoveredModal from '../components/devices/TryAllDiscoveredModal';
type DeviceSortKey = 'name' | 'ip_address' | 'model' | 'ros_version' | 'status' | 'last_seen';
type DeviceSortKey = 'name' | 'ip_address' | 'model' | 'serial_number' | 'ros_version' | 'status' | 'last_seen';
type DiscoveredSortKey = 'identity' | 'address' | 'mac_address' | 'seen_by' | 'discovered_at';
type SortDir = 'asc' | 'desc';
@@ -85,9 +85,13 @@ function TypePill({ type }: { type: Device['device_type'] }) {
function CpuValue({ value }: { value: number | undefined }) {
if (value == null) return <span className="mono text-[11px]" style={{ color: 'var(--ink-4)' }}>—</span>;
// The series behind this is a 5-minute mean from InfluxDB, so a raw value is
// routinely something like 1.3333333333333333. Round for display only — the
// sparkline keeps the unrounded points so its curve stays smooth.
const shown = Math.round(value);
const color = value > 70 ? 'var(--bad)' : value > 40 ? 'var(--warn)' : 'var(--accent)';
return (
<span className="mono num-tab text-[12px] font-medium" style={{ color }}>{value}%</span>
<span className="mono num-tab text-[12px] font-medium" style={{ color }}>{shown}%</span>
);
}
@@ -193,7 +197,8 @@ export default function DevicesPage() {
if (search &&
!d.name.toLowerCase().includes(search.toLowerCase()) &&
!d.ip_address.includes(search) &&
!d.model?.toLowerCase().includes(search.toLowerCase())
!d.model?.toLowerCase().includes(search.toLowerCase()) &&
!d.serial_number?.toLowerCase().includes(search.toLowerCase())
) return false;
if (statusFilter === 'online' && d.status !== 'online') return false;
if (statusFilter === 'offline' && d.status !== 'offline') return false;
@@ -210,6 +215,7 @@ export default function DevicesPage() {
case 'name': return x.name || '';
case 'ip_address': return x.ip_address || '';
case 'model': return x.model || '';
case 'serial_number': return x.serial_number || '';
case 'ros_version': return x.ros_version || '';
case 'status': return x.status || '';
case 'last_seen': return x.last_seen ? new Date(x.last_seen).getTime() : 0;
@@ -402,6 +408,7 @@ export default function DevicesPage() {
{ key: 'name', label: 'DEVICE', w: null },
{ key: null, label: 'TYPE', w: 72 },
{ key: 'model', label: 'MODEL', w: null },
{ key: 'serial_number', label: 'SERIAL', w: 130 },
{ key: 'ip_address', label: 'IP ADDRESS', w: null },
{ key: 'ros_version',label: 'ROS', w: 110 },
{ key: null, label: 'CPU', w: 110 },
@@ -465,6 +472,15 @@ export default function DevicesPage() {
<td className="px-4 py-[12px]">
<span className="mono text-[11.5px]" style={{ color: 'var(--ink-2)' }}>{device.model || '—'}</span>
</td>
<td className="px-4 py-[12px]">
<span
className="mono text-[11.5px] block truncate"
style={{ color: 'var(--ink-3)', maxWidth: 130 }}
title={device.serial_number || undefined}
>
{device.serial_number || '—'}
</span>
</td>
<td className="px-4 py-[12px]">
<span className="mono num-tab text-[12px]" style={{ color: 'var(--ink-2)' }}>{device.ip_address}</span>
</td>
frontend/src/version.ts
+1 -1
View File
@@ -1 +1 @@
export const APP_VERSION = 'v0.20.0 Beta';
export const APP_VERSION = 'v0.20.1 Beta';