Proposal: a received-message log in signalk-server core (DSC first)

signalk-message-log.md

Proposal: A received-message log in signalk-server core (DSC first)

Working draft for discussion, off the back of Teppo's suggestion (Discord) that the signalk-dsc plugin's call log become a first-class server feature.

Goal

A first-class log of received digital messages in signalk-server: a data model the server owns, a query/manipulate REST API in the likeness of the History API, and (later) a Kip-style panel like the autopilot control panel. DSC is the first and, for v1, the only message type — but the model is generic so NAVTEX (and AIS safety/addressed text) can slot in later without reshaping it.

This keeps the regulatory "log every distress/urgency/safety call" record (SOLAS Ch. IV, 47 CFR §80.409) as a property of the server, not a plugin.

Scope (settled)

• Generic envelope, DSC-only implementation in v1. Design the schema to hold any received digital text/structured message; ship only the DSC type.
• In scope as future types: NAVTEX (the validating second case), AIS safety / addressed text messages.
• Out of scope — the line where it gets too application-specific: media streams and two-way comms. Radiofax (images) and NOAA Weather Radio (audio) are resources, not log entries; SMS is interactive, not a received feed. Rule of thumb: structured/text digital messages in; media and conversations out.
• No provider delegation in v1, but the storage seam is designed in (below).

Architecture

Mirrors the existing provider/registry APIs (autopilot, history, resources, notifications): the server owns the data model + REST API + panel.

 transport parsers              server core                          clients
 (separate repos)
 ─────────────────              ─────────────────────────────        ─────────
 nmea0183-signalk ─structured─▶ DSC adapter ─app.logMessage()─▶       REST API ─▶ Kip panel
 n2k-signalk        call (typed)              │                       /v2/api/…   3rd-party
   │                                          ▼
   └─ position + notification deltas ─▶ MessageLog ─▶ SQLite (node:sqlite)
      (current PRs, unchanged)              │
                                            └─▶ NotificationManager (live, ackable)

• Parsers stay "bytes → structured call." They keep their non-lossy extraction in the lib and expose the full parsed call as a typed return — they do not persist, and they do not own log schema. Their existing delta/notification output (SignalK/n2k-signalk#332, nmea0183-signalk#336 & #337) is unchanged; what Option 2 adds is surfacing the structured fields the parsers already compute.
• Core DSC adapter consumes that typed parser output and submits entries via the ingestion API. This is the only DSC-aware code in core; the log subsystem itself is type-agnostic.
• Message-log subsystem persists submitted entries and raises notifications for the actionable ones.
• REST API in core, History-API-shaped.
• UI panel — separate, later.

Data model

A common envelope + a type-specific payload:

Field	Type	Notes
id	string	stable id
type	string	dsc (only value in v1)
receivedAt	ISO 8601	server receive time
sourceRef	string	$source — which connection/device heard it (Teppo's ask)
transport	string	nmea0183 | nmea2000
priority	string	distress | urgency | safety | routine (DSC category)
sender	object	station identity, e.g. { mmsi, name? }
subject	object?	vessel in distress on a relay, e.g. { mmsi }
position	object?	{ latitude, longitude } when reported
summary	string	voice-sized one-liner
payload	json	type-specific (DSC: nature, workingChannel, utcTime, ack, ownShip snapshot…)
raw	string	original sentence(s) / PGN
notificationId	string?	link to the live notification, if any
disposition	object	{ acknowledgedAt?, clearedAt? } audit trail

The envelope columns are the queryable surface; everything type-specific lives in payload so a new type needs no schema change.

Storage — node:sqlite, behind a seam

Decision (start small, fan out later): core owns a node:sqlite store directly, behind a thin interface so a Postgres-class backend can register later. Ship only the sqlite default in v1.

• node:sqlite is available on the server's Node floor (>=22) with no native build step — the real win on a Pi vs. better-sqlite3. This would be core's first built-in DB, so it's a deliberate precedent. Still flagged experimental upstream, so pin behavior and keep the seam thin.
• DB file under app.getDataDirPath(); WAL mode (power-loss recovery on a boat), foreign keys on, a schema_version/migrations table. (Mirrors the patterns in the community signalk-database plugin — but core uses node:sqlite directly rather than depending on a third-party early-dev plugin.)
• Not the History Provider seam: that interface is time-series numeric (getValues/getContexts/getPaths, aggregate buckets) — the wrong shape for discrete message records. So "reuse the user's existing DB" realistically means Postgres-class (e.g. signalk-postgsail), not the common InfluxDB.

The seam:

interface MessageLogStore {
  append(entry: MessageLogEntry): Promise<MessageLogEntry>
  get(id: string): Promise<MessageLogEntry | undefined>
  query(filter: MessageLogQuery): Promise<MessageLogEntry[]>
  update(id: string, patch: DispositionPatch): Promise<MessageLogEntry | undefined>
}

Default: SqliteMessageLogStore. v2+: allow a plugin to register an alternative.

Indexes: receivedAt, type, priority, sender.mmsi.

Ingestion — core API (settled: Option 2)

Decision: core exposes an ingestion API — app.logMessage(entry) — and a small core DSC adapter feeds it from the parser output. We do not introduce a communications.* path that the log subscribes to.

// @signalk/server-api
app.logMessage(entry: MessageLogEntryInput): Promise<MessageLogEntry>

Why this over a structured communications.* delta:

• One door in. Manual voice-call entries and future NAVTEX / AIS-text messages never come through the NMEA parsers — they have no delta to subscribe to. A path-subscription model would only ever cover the parser-fed subset and force a second ingestion path for everything else. A single logMessage() serves all producers.
• Cheap to evolve. communications.* would be spec-level path vocabulary — expensive to reverse once it ships in deployed parser releases. An internal ingestion API can change without breaking on-the-wire contracts.
• Discrete events ≠ current state. Deltas model the current value of a path; received calls are discrete events. Forcing them through the path/delta model is awkward — the same reason notifications aren't "just deltas."

Parser contract (confirmed with Marc): the parser libs keep the full structured-call extraction non-lossy in the lib and deliver it via a typed return, not a stateful path. Parsers stay "bytes → structured call"; core owns the log schema and persistence. The core DSC adapter is the glue that maps a parsed call onto a logMessage() submission.

REST API (History-API likeness)

Under /signalk/v2/api/…, wired as a CommunicationsApi class in src/api/index.ts like the other APIs, with an OpenAPI doc.

GET  /signalk/v2/api/communications/messages
       ?from=&to=&type=&priority=&sender=&limit=&order=
GET  /signalk/v2/api/communications/messages/:id

• Read/query is anonymous under allow_readonly (like resources), so chartplotters and the briefing can read the log without a token.
• Disposition (ack/clear) is not a new write surface — it flows through the existing notifications ack (below) and is reflected onto the entry.
• (Path naming — open: see below.)

Notifications integration

The server already tracks notification state + ack (/signalk/v2/api/notifications, …/:id/acknowledge, …/acknowledgeAll, NotificationManager). The log must not reinvent that.

• distress / urgency / safety → raise a notification via NotificationManager; store its id on the log entry. Ack/clear flows through the existing notifications API; the log entry's disposition is updated to mirror it (audit trail).
• routine → log entry only, no notification.
• Split of concerns: notifications = "needs attention now" (live, ackable); log = "what was received" (permanent, queryable). The regulatory record is satisfied regardless of whether anyone ack'd.

Phasing (one logical change per PR)

1. ✅ Parser fixes — n2k#332, nmea0183#336, nmea0183#337.
2. MessageLogStore interface + SqliteMessageLogStore (node:sqlite) + envelope types in @signalk/server-api.
3. app.logMessage() ingestion API + the DSC envelope/payload type.
4. CommunicationsApi REST module + OpenAPI.
5. Parser libs expose the full structured call (typed return) + core DSC adapter wiring it to logMessage().
6. Notifications integration (raise + disposition mirroring).
7. Kip / admin UI panel — separate effort.

Open questions

The ingestion question is settled (Option 2, above). Two smaller ones remain:

• Endpoint naming. communications/messages filterable by type, vs communications/dsc, vs a top-level messages. Leaning communications/messages — does the resource shape want to live elsewhere?
• Routine-entry disposition. Ack covers anything that raised a notification (distress/urgency/safety). Routine calls raise none, so they have no disposition lifecycle. Do we add a plain clear/dismiss on the log for them (mark-as-reviewed), or do routine entries stay a stateless permanent record? Leaning stateless, with "reviewed" implicit — open to the audit angle wanting something explicit.