The monitor watches every source for the four things that go wrong before a record does: row volume, freshness, structure and value distributions.
| Signal | What is watched | A problem | What the monitor does |
|---|---|---|---|
| Volume | Rows and change events per interval, against the source's learned range | A 95% drop in change events at 02:14 | Pauses the dependent feeds, pages the engineer, tells the consumers; resumes and replays the backlog when the source recovers |
| Freshness | The age of the latest change per source and per view, against the contracts | A view older than its six-hour contract | Notifies the consumer; holds the feed if the contract says so |
| Structure | Columns, types and lengths at the source, against the registered shape | A column type changed, a column added | Catalogues the change, updates the read, lists the views that use the column, notifies the application owner if a type change breaks a rule |
| Values | Distributions of the weighted fields, against the learned baseline | 400 negative quantities in an hour where two is normal | Opens a ticket with the affected records and the root cause traced through lineage |
A fixed threshold fires on every month-end order surge and every Saturday dip, and the people who receive those alerts learn to ignore them. The monitor learns each source's rhythm over a rolling window: weekly seasonality, calendar effects, campaign periods. An event inside the learned pattern stays silent; an event outside it opens a ticket. On the showcase estate the change took alerts from 41 a week to 6, and the 6 were real.
The learned pattern is shown, so a suppressed event is explainable: 'order volume +82% on the last trading day, seen in 11 of the last 12 months'. A steward can mark a suppressed event as a true anomaly, and the pattern learns from that too.
Root cause runs through lineage. A value anomaly on a golden record is traced back through the rule run, the source, the module and the rows: on the showcase estate, 847 negative quantities were traced in 28 seconds to a returns module update at one site, affecting three product categories. The ticket carries the trace and the recommended actions.
Consumers are told by contract: the systems reading the affected views receive the notice with the records and the dimension, and the feed is held if the contract says so. Nothing stale is served in silence.
Forty-four minutes from detection to close, and zero stale records served: the feed was held instead. Had the feed run, the morning dashboards would have shown the previous day's data to two hundred people. Showcase estate.
It uses the same signals and the same lineage; the difference is what it protects. The Backbone watches pipelines; the monitor here watches the sources of golden records and enforces the contracts on the golden views.
No level of the monitor writes data. Its actions are pausing and resuming feeds, notifying, and opening tickets; the corrector and a person do the changing.
Ninety days by default, rolling; a source with strong annual seasonality can be given a longer window by the owner.
A notice with the view, the dimension, the records affected and the expected recovery, through the contract's channel: a message, a ticket in the consumer's system, or the API's status field.
Bring one domain and two systems. In a week the sources are profiled, the ten costliest fields are under rules, the first corrections are written back with their acknowledgements and the publish gate is on.