Monitoring
Running a sensor network means noticing when something needs attention — a membership request waiting on a curator, a station that went quiet, an integration that failed mid-pipeline, or a datastream reading outside expected bounds.
Release 3 puts that work in a monitoring domain on the Platform API, with a monitor worker doing the evaluation in the background. Organizations configure rules for what to watch. When a rule fires, Dendra opens an alert that stays on the dashboard until it clears or someone resolves it. Whether anyone also gets email, SMS, or Slack depends on their notification preferences.
For how the worker is scheduled, see Job Scheduler and Workers.
Rules, Alerts, and Catalog Types
Section titled “Rules, Alerts, and Catalog Types”Platform deploy config (api.toml) defines the catalog: rule types, alert types, and notification types. Organizations then create rule instances — enabled rules with a scope, severity, chosen alert type, and optional parameters.
A rule type says what kind of signal it listens for and which alert types it may produce. A rule is the live instance for an org (or station, datastream, or integration). An alert is the open concern that shows up until it clears or someone resolves it.
Rules listen on different evaluator planes:
| Plane | What triggers it | Examples |
|---|---|---|
| Change event | Metadata changes | Membership requested or granted, annotation pending |
| Pipeline message | Integration pipeline outcomes | Runtime errors, stage failures |
| Scheduled | Timer ticks | Station offline, missing data |
| Data notice | New timeseries data | Thresholds, tolerance limits, no-change |
How an Alert Opens
Section titled “How an Alert Opens”The monitor worker loads enabled rules, evaluates incoming signals, and emits a condition event — either FIRING or CLEARING. That feeds a short pipeline:
---
config:
flowchart:
nodeSpacing: 40
rankSpacing: 50
padding: 16
---
flowchart LR
Signal["Incoming signal"]
Eval["Evaluate rules"]
Cond["Condition event"]
Alert["ReportAlert"]
Notify["Notify if new"]
Signal --> Eval --> Cond --> Alert --> Notify
On FIRING, the alert engine looks for an existing open alert with the same correlation key. If none exists, it creates one. If one already exists, it updates evidence and counts — no second notification. On CLEARING, the matching open alert is resolved automatically. Users can also acknowledge or resolve alerts from the Main App.
Optional side effects on a rule (for example creating an annotation) run after the alert update. Sending a notification is never treated as a side effect.
Who Gets Notified
Section titled “Who Gets Notified”Nothing is sent until notification preferences say it should be. For most alerts, that happens when a new alert opens: the worker looks up organization preferences for likely recipients, matches their subscriptions, and delivers. Repeat firings on an already-open alert do not trigger another send.
A few messages skip the alert path entirely. Things like a membership welcome are handled as change events and use system-wide preferences (not tied to an organization).
People can subscribe by notification type, alert type, or a specific rule. Channels are email, SMS, and Slack, either immediately or rolled into a digest.
An Example Walkthrough
Section titled “An Example Walkthrough”Say someone requests to join an organization. A change event lands, the monitor worker finds an enabled membership-requested rule, and evaluation emits FIRING. ReportAlert opens (or updates) an alert keyed to that request. Because this is a new open alert, the notify stage loads curator and admin preferences, renders the email template, and records a delivery.
Later the request is approved. Evaluation emits CLEARING for the same correlation key, the alert resolves, and no second “request pending” email goes out. A separate membership-granted path can open its own alert and notify subscribers who care about grants.