Pipeline Tests

Two different things gate a Data Conductor pipeline, and only one of them is available today.

Status What it does
Assertions Available A query plus an expectation, attached to a step. Warns, pauses or stops the run.
End-to-End Tests In development Input fixtures injected in place of live data, with assertions checked against the result.

If you are looking for a way to stop bad data reaching production now, you want Assertions. The rest of this page describes where end-to-end tests are going, so the distinction is clear when they land.

What end-to-end tests will add

An assertion checks the data a step actually produced, against whatever was in the database at the time. That is the right check for most things and it has one limit: the result depends on live data, so a pipeline that is correct today can pass and the same pipeline can fail tomorrow for reasons that have nothing to do with its logic.

An end-to-end test removes that variable. It pairs input fixtures — rows you control — with assertions about what the pipeline should turn them into. Same input every run, so a failure means the logic changed.

The card for it is on the Data Tools hub, marked coming soon.

Writing a check worth having

This applies to assertions now and to fixtures later. Assert the things that would actually hurt if they were wrong:

  • Row counts within an expected range, not an exact number that will drift and leave you editing the test every week.
  • Nulls absent from columns that must never be null.
  • Totals reconciling against a known control figure.
  • Keys unique where you expect uniqueness.

A check that only confirms the pipeline did not error tells you nothing the run status does not already say.

A worked example

Stark Industries rebuilds ANALYTICS.MARTS.MRR_BY_MONTH every morning. Three things would be quietly wrong if the rebuild went wrong, and none of them would fail the run on their own:

  • the same month and plan appearing twice, which doubles revenue in every chart downstream
  • a null MRR, which most dashboards render as a gap rather than an error
  • a negative OUTSTANDING, which means more was paid than billed and usually points at a join fanning out

All three fit in one assertion, because an assertion is a query plus "expect no rows" — so several checks can share one by returning a row per problem found:

SELECT 'duplicate month+plan' AS check_name, COUNT(*) AS offending_rows
FROM (SELECT month, plan
      FROM ANALYTICS.MARTS.MRR_BY_MONTH
      GROUP BY month, plan HAVING COUNT(*) > 1)
UNION ALL
SELECT 'null mrr', COUNT(*)
FROM ANALYTICS.MARTS.MRR_BY_MONTH WHERE mrr IS NULL
UNION ALL
SELECT 'negative outstanding', COUNT(*)
FROM ANALYTICS.MARTS.MRR_BY_MONTH WHERE outstanding < 0

On a healthy rebuild every offending_rows is 0. Attach it to the final step with timing after and policy stop, because unlike the collections example in Assertions, each of these means the pipeline itself produced something wrong — there is no point building reports on it while someone investigates.

Naming each check in the result is what makes this worth doing as one assertion rather than three. A failure tells you which invariant broke without opening the query.