feat(commands): run one command across a fleet, in waves (v0.23.27-beta)

Closes #118. Unblocked by #110, which is what makes SSH on sixty devices
practical.

Arbitrary RouterOS console commands over SSH, because `:put`, `:foreach` and
`/interface print` are console syntax that the binary API cannot express, and
because a curated command library cannot cover "add a channel to LACP" or
"rewrite a WireGuard endpoint" — the things people actually asked for.

**Waves are the feature, not the safety rail around it.** A shell loop already
runs a command on many hosts, costs nothing, and prints errors as they happen.
What it cannot do is stop. Devices run together within a wave, waves run strictly
in sequence, and halt-on-failure ends the run before a mistake becomes sixty
mistakes. Without that, this would be a worse bash.

Proved on hardware rather than argued: a bad command on two devices, one per
wave, failed on the first and left the second `skipped`. RouterOS reports most
errors in output text rather than an exit code, so output is read for failure
markers — otherwise a rejected command and a successful one are indistinguishable
and the run would sail on. Conservatively: only unambiguous markers count, since
misreading ordinary output would halt a run that was working.

Change Guard wraps each device by default, so one that stops answering after the
command restores itself. It is one click to turn off, because a sharp tool that
refuses to cut is not a tool — the requester's argument, and a fair one. The
combination that gets an explicit acknowledgement is a command that can sever
management *with the guard disabled*; that is the only case where nothing catches
a mistake.

A preview names devices, wave assignment, and the reasons a command looks capable
of cutting management access. It warns, never blocks. Devices with no SSH
credential are listed before the run rather than producing a wave of identical
authentication failures.

Per-device output is captured and kept, because "it failed" without the device's
own words is not actionable, and because some commands exist purely for output.

Verified end to end against real hardware: a read-only command across two
devices, halt-on-failure stopping a bad command after one device, and a genuine
write applied under Change Guard, confirmed, then reverted — with both devices
returned to their original configuration afterwards.

Also adds services/sshExec.ts, shared with the key work: it selects a verified
key or a password, and throws rather than swallowing. That last part is
deliberate — a swallowed error is exactly how the data-cap feature spent a week
doing nothing while looking healthy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2GT-Rich
2026-09-02 14:20:02 -07:00
co-authored by Claude Opus 5
parent 19e2d914f5
commit 0e364b1558
14 changed files with 844 additions and 5 deletions
+32
View File
@@ -387,6 +387,38 @@ export const sshKeyApi = {
'/devices/ssh-keys/deploy-all', { confirm: true }, { timeout: 600_000 }),
};
// ─── Bulk commands (#118) ─────────────────────────────────────────────────────
export interface CommandRunSummary {
id: number; name: string; command: string; status: string;
wave_size: number; halt_on_failure: boolean; use_change_guard: boolean;
total: number; succeeded: number; failed: number; created_at: string;
}
export interface CommandRunDevice {
id: number; device_id: number; device_name: string; ip_address: string;
wave: number; status: string; output: string | null; error: string | null;
guarded: boolean; auto_reverted: boolean;
}
export interface CommandRunDetail extends CommandRunSummary {
devices: CommandRunDevice[];
}
export const commandsApi = {
preview: (command: string, device_ids: number[], wave_size: number) =>
api.post<{
command: string; risky: boolean; reasons: string[]; wave_size: number; waves: number;
devices: { id: number; name: string; wave: number; status: string }[];
unreachable: { id: number; name: string }[];
}>('/commands/preview', { command, device_ids, wave_size }),
createRun: (body: Record<string, unknown>) => api.post<{ id: number }>('/commands/runs', body),
listRuns: () => api.get<CommandRunSummary[]>('/commands/runs'),
getRun: (id: number) => api.get<CommandRunDetail>(`/commands/runs/${id}`),
cancelRun: (id: number) => api.post<{ message: string }>(`/commands/runs/${id}/cancel`),
};
export const devicesApi = {
list: () => api.get<Device[]>('/devices'),
discovered: () => api.get<DiscoveredDevice[]>('/devices/discovered'),