gustavosantos/MIKROTIK-RMM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: place log events in real time, reset the data cap on send, export runs (v0.23.29-beta)

Browse Source 
Three reports from this morning.

**Event timestamps were not the times the events happened.** RouterOS writes log
times in the device's own timezone and never says so, and it uses more than one
format. The parser handled `sep/03 08:37:38` and `08:37:38` but labelled both
UTC, shifting every event by the device's offset — reported as an event log
seven hours out from the device page. It did not handle
`2026-09-03 08:37:38` at all, which is what current RouterOS actually returns
over the API, so those fell through to the collection time instead: plausible on
a live poll, badly wrong for a backlog, where a thousand historical lines all
arrive stamped "now".

Devices now report their clock, and timestamps are converted from device-local
to UTC through it, with the numeric gmt-offset as a fallback and daylight saving
handled by the runtime rather than a fixed number. Where the zone is unknown the
parser returns null and the caller falls back to collection time, deliberately —
a confidently wrong timestamp is worse than an openly approximate one.

Verified on hardware: 18 of 18 events in a steady-state window now store exactly
seven hours ahead of what the device printed, which is the offset the device
itself reports. Existing rows are left alone; they were written by three
different code paths and cannot be corrected uniformly.

**The data-cap counter kept climbing after the SMS was sent.** Reported from the
field, and correct: the message is what zeroes the carrier's allowance — it is
the alternative to waiting for midnight — so ours has to follow. Left as it was,
two things went wrong: the bar counted against an allowance that no longer
applied, and usage stayed above the threshold, so the rule would re-fire on
every cooldown for the rest of the day. Both send paths now restart the count.

**Command runs export to CSV**, asked for on #118 for archiving and audit.
Output is quoted so multi-line device responses survive, and timestamps are ISO
rather than the driver's default string form, which no spreadsheet parses. The
download goes through the authenticated client rather than a plain link, because
a link cannot carry a Bearer header and would simply 401.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2GT-Rich
2026-09-03 08:50:01 -07:00
co-authored by Claude Opus 5
parent cce1f88a45
commit 324ceba676
13 changed files with 374 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/src/services/mikrotik/DeviceCollector.ts
+27 -36
View File
@@ -17,6 +17,7 @@ import {
parseLteMonitor, overallQuality, detectChanges, type LteStatus,
} from '../../utils/lte';
import { bytesSince, periodKey, shouldSend } from '../../utils/dataCap';
import { parseDeviceLogTime } from '../../utils/deviceTime';
import { BackupService } from '../BackupService';
import { alertService } from '../AlertService';
@@ -195,6 +196,8 @@ export class DeviceCollector {
const identity = await this.client.execute('/system/identity/print');
const resource = await this.client.execute('/system/resource/print');
const routerboard = await this.client.execute('/system/routerboard/print').catch(() => [] as Record<string, string>[]);
// Needed to place log timestamps correctly — they are device-local (#117).
const clock = await this.client.execute('/system/clock/print').catch(() => [] as Record<string, string>[]);
const info = resource[0] || {};
const rb = routerboard[0] || {};
@@ -212,6 +215,8 @@ export class DeviceCollector {
serial_number = COALESCE($3, serial_number),
firmware_version = COALESCE($4, firmware_version),
ros_version = COALESCE($5, ros_version),
time_zone_name = COALESCE($7, time_zone_name),
gmt_offset = COALESCE($8, gmt_offset),
-- An update that has landed is no longer pending. The dedicated
-- firmware check only runs once a day, so without this a device stays
-- flagged "update available" for up to 24 hours after it has already
@@ -224,7 +229,8 @@ export class DeviceCollector {
END,
updated_at = NOW()
WHERE id = $6`,
[identityName, model, serial, firmware, rosVersion, this.device.id]
[identityName, model, serial, firmware, rosVersion, this.device.id,
clock[0]?.['time-zone-name'] || null, clock[0]?.['gmt-offset'] || null]
);
} catch (err) {
console.error(`[${this.device.name}] Failed to collect system info:`, err);
@@ -843,41 +849,19 @@ export class DeviceCollector {
}
}
/**
* A log timestamp as a real instant, using the device's own clock.
*
* Falls back to the collection time only when the device timezone is unknown
* or the format is unrecognised — approximate, but never confidently wrong by
* a whole timezone, which is what the previous version was (#117).
*/
private parseLogTime(timeStr: string): Date {
// RouterOS log time formats:
// "jan/01 00:00:05" — month/day only (no year) → default to current year
// "jan/01/2024 00:00:05" — full date with year
// "00:00:05" — time only (today, e.g. device uptime < 1 day)
if (!timeStr) return new Date();
try {
if (timeStr.includes('/')) {
const spaceIdx = timeStr.lastIndexOf(' ');
if (spaceIdx === -1) return new Date();
const datePart = timeStr.substring(0, spaceIdx);
const timePart = timeStr.substring(spaceIdx + 1);
const parts = datePart.split('/');
const monthStr = parts[0];
const day = parts[1];
if (!monthStr || !day) return new Date();
const year = parts[2] !== undefined ? parts[2] : String(new Date().getFullYear());
const months: Record<string, string> = {
jan: '01', feb: '02', mar: '03', apr: '04', may: '05', jun: '06',
jul: '07', aug: '08', sep: '09', oct: '10', nov: '11', dec: '12',
};
const month = months[monthStr.toLowerCase()] || '01';
const d = new Date(`${year}-${month}-${day.padStart(2, '0')}T${timePart}Z`);
if (isNaN(d.getTime())) return new Date();
return d;
} else {
// Time only (today)
const today = new Date().toISOString().split('T')[0];
const d = new Date(`${today}T${timeStr}Z`);
if (isNaN(d.getTime())) return new Date();
return d;
}
} catch {
return new Date();
}
const d = this.device as unknown as { time_zone_name?: string | null; gmt_offset?: string | null };
const parsed = parseDeviceLogTime(timeStr, {
timeZoneName: d.time_zone_name, gmtOffset: d.gmt_offset,
});
return parsed ?? new Date();
}
private mapLogSeverity(topics: string): string {
@@ -3088,10 +3072,17 @@ export class DeviceCollector {
}, now);
let sentAt: Date | null = rolled ? null : (rule.last_sent_at ? new Date(rule.last_sent_at) : null);
// Sending the reset SMS zeroes the carrier's counter — that is what the
// message is for, and it is the alternative to waiting for midnight. Our
// counter has to follow, or two things go wrong: the bar keeps climbing
// past an allowance that no longer applies, and usage stays above the
// threshold so the rule re-fires on every cooldown for the rest of the day.
let periodBytesAfter = periodBytes;
if (decision.send) {
try {
await this.sendSms(interfaceName, rule.phone_number, rule.message);
sentAt = now;
periodBytesAfter = 0;
await this.logDataCapSend(interfaceName, 'threshold', rule.phone_number, periodBytes, true, null);
console.log(`[DataCap] ${this.device.name}/${interfaceName}: reset SMS sent to ${rule.phone_number}`);
} catch (e) {
@@ -3105,7 +3096,7 @@ export class DeviceCollector {
SET period_key = $2, period_bytes = $3, last_rx = $4, last_tx = $5,
last_sent_at = $6, updated_at = NOW()
WHERE id = $1`,
[rule.id, key, periodBytes, counters.rxBytes, counters.txBytes, sentAt]
[rule.id, key, periodBytesAfter, counters.rxBytes, counters.txBytes, sentAt]
);
} catch (e) {
console.error(`[DataCap] ${this.device.name}: tracking failed:`, (e as Error).message);