Security

Most of our users are the person who decides whether a tool is allowed in, as well as the person who uses it. This page is written for that: direct answers to the security and privacy questions that decide it, no compliance theatre, and nothing claimed that is not actually true today. Where something is still in progress it says so.

Where does my data actually go?

Data Conductor runs SQL inside your database. It is not a warehouse and it does not hold a copy of your data. A pipeline step connects to the database you pointed it at, runs your SQL there, and the results come back to your browser.

What we store about a run is the bookkeeping, not the data: which step ran, when, how long it took, whether it succeeded, how many rows it affected, and the error if it failed.

One honest caveat on that last item. Error text comes from your database verbatim, because a sanitised error is useless for debugging — and a database error message can quote a value that caused it, such as the duplicate key in a constraint violation. So error records can contain fragments of data, by the nature of what an error is.

Moving data between systems

A transfer stages through object storage, and that bucket is yours. A transfer requires a storage connector you attach with your own credentials, and the export step writes Parquet into your bucket before the load step reads it into the destination. Retention, encryption settings and access on that staging data are governed by your own bucket policy, not by ours.

Uploading files

Files you upload for ingestion are the one case where data lands in our infrastructure. They go to a staging bucket, and that bucket has a lifecycle rule that deletes objects automatically after one day. The application never relies on deleting them itself.

Does my data go to an AI model?

This is the question we get asked first, and it deserves a precise answer rather than a reassuring one.

What is sent to the model:

  • Your schema metadata — table names, column names, data types, and any descriptions you or the indexer have written.
  • The SQL currently in your editor, when you are asking about it.
  • The messages you type.

What is not sent: the rows your queries return. Query results are returned to your browser and are not part of any prompt the platform builds.

The exception, which is yours to make deliberately: a pipeline can contain an agent step, and a step configured to return its rows into the pipeline can hand that data to an agent step you have built. If you build that, your data goes to a model — because you asked it to. Nothing does it implicitly.

If you build an agent step, which model sees it?

The one your own credential points at. An agent step runs against a connection you added yourself, using your API key with the provider you chose. Today that can be Anthropic Claude, Google Gemini, OpenAI, Perplexity or xAI Grok.

Perplexity is the one worth reading twice. Its online models answer with live web context, which means a lookup on the open web as part of answering — not inference alone. That is why you would choose it, and it is a different data-handling question from the others. Pick it knowingly.

That API key is protected exactly like a database password — the same per-secret encryption and the same per-organisation key described in the next section. A model provider's key is a credential, and nothing about it is treated more loosely.

An agent step is sent what its prompt asks for by name, not your result set. This is the part most people assume works the other way around:

  • The prompt is the one saved on the agent, with its {{placeholders}} filled in.
  • Each value comes from an input you declared — one named field, from one named upstream step — or from an environment variable.
  • A field nobody declared is not sent. The rows an upstream step produced do not travel to the model because they happen to be in the same pipeline.

Agent steps have no tools. An agent running inside a pipeline cannot query a database, call an API or act on anything. It is given a prompt and returns an answer. There is nobody present during a scheduled run to approve an action, so we do not give it any to take.

How are my database credentials stored?

Encrypted at rest, with a separate key per organisation.

  • Each secret value — passwords, private keys, access tokens, storage secrets — is encrypted with AES-256-GCM before it is stored.
  • The key that encrypts them is unique to your organisation.
  • That key is itself wrapped by a master key held in Google Cloud KMS. The master key never leaves KMS; we cannot export it.
  • Non-secret fields such as hostnames stay readable, so the platform can still tell you which database a pipeline points at without unwrapping anything.

Only an Owner can manage credentials. Credential management is granted to the Owner role alone — a regular user on your account cannot add, edit or view connection settings.

The agent never receives credential values. When the assistant lists your connectors to reason about them, the configuration is stripped out before it reaches the model. It can know that a Snowflake connection called "Analytics" exists; it cannot know anything inside it.

Could the assistant be tricked into reading another customer's data?

No — and not because the assistant is well behaved. Because it has no tool that could do it.

The assistant can only act through tools it has been given, and the services that read across organisations are not given to it at all. They are absent from its catalogue, so there is no call to talk it into making. Those same services are additionally gated on the server, and the browser plays no part in the check — there is no flag or setting on your side that changes what anyone can reach.

This is the one place we deliberately break our own rule. Normally anything you can do in the UI is also something the assistant can do, and a gap between the two counts as a bug. Operator views are the exception on purpose, because a tool that can read every organisation's data would be the single most valuable thing on the platform to attack — so we do not build one. A prompt injection cannot reach a capability that does not exist.

What can I review afterwards?

Every tool the assistant invokes on your behalf is recorded and reviewable for your organisation, filterable by tool and time. If you want to know what the agent actually did rather than what it said it did, that is the record.

Pipeline run history keeps per-step status, timings, row counts and the full error output for failures.

How does it reach my database?

Data Conductor connects outbound to you. You do not open your database to the internet or install an agent inside your network.

Outbound connections leave from a small fixed set of addresses, which you can allowlist on your database or firewall. The current addresses are listed on the Organization Settings page in the app — read them there rather than from documentation, so you are always allowlisting the live set.

Does Data Conductor write anything to my database?

Almost never, and never without you asking for it.

Reading your schema, profiling tables and running a query are reads and transient compute. Nothing is left behind. A pipeline writes only where your own SQL tells it to — the CREATE TABLE or INSERT is yours, and it goes where you wrote it.

There is one object Data Conductor creates that you did not write, and it only appears if you use migrations:

_data_conductor_migrations — created in the database and schema your connector points at, the first time you apply a migration in PROD. (Those have to be set on the connector; on Snowflake a session opens with neither unless the credential supplies them.) It is how a migration runs once rather than every night. One row per applied migration: which step, which version, which environment, a checksum of the SQL, when it was applied and by whom.

Three things about it worth knowing:

  • You are asked first. The first PROD migration against a connector stops and tells you the exact table name it wants to create, before anything runs, quoting what your database said when it looked for it. Decline and nothing happens — not the table, not the migration. If the connector's role cannot reach that schema, you get an error naming the setting to change rather than an offer to create a table that cannot be created.
  • It is never created in QA, or any other non-PROD environment. Those run your DDL every time and track nothing.
  • It is an ordinary table. Read it, audit it, grant on it, back it up. If you drop it, Data Conductor asks again the next time and then rebuilds it — though it will have forgotten what was already applied, so migrations that have run once may run a second time.

If you never mark a step as a migration, this table never exists.

What happens if something is lost?

The platform database — your organisation, connections, pipelines, indexed schema and audit trail — is a regional Cloud SQL instance with a standby in a second zone, so a zone failure does not take it offline.

High availability is not a backup, and the two solve different problems: a standby copies a mistaken DROP as faithfully as it copies anything else. So the instance also takes an automated daily backup at 03:00 UTC, retained for 14 days, stored in the US multi-region — outside the region the instance runs in.

On top of that, point-in-time recovery keeps 7 days of transaction logs, so recovery is not limited to the state at 03:00: the database can be restored to a chosen moment within that window. That is the difference between losing a day's work to a mistake and losing the minutes around it.

Note what this does not cover. It is the platform database, not your warehouse: your own data stays in your systems (see Where does my data actually go?) and its backups remain yours to run. Nor does it cover the sample sandbox database given to new organisations, which is deliberately disposable — see below.

How is the sample database isolated?

Every new organisation is given a Sample Database (Pagila) so the first useful minute needs no credential of your own. It is a real Postgres database, and the isolation around it is worth stating because it is the one database we run on your behalf.

  • It is not on the platform database. Sandboxes live on a separate Cloud SQL instance. The platform instance holds stored connection secrets, so a role that can write SQL must not exist on it at all.
  • One database per organisation, created as its own copy of a template rather than a shared schema. Nothing is co-tenanted.
  • One login per organisation, with a generated password. CONNECT is revoked from PUBLIC on that database and granted only to that organisation's role, so one sandbox login cannot reach another organisation's sandbox even on the same instance.
  • It expires after 30 days. On expiry the database and the role are dropped, open connections are terminated, and the sample file issued with it is deleted from storage.

Treat it as a demo, not a home: it is disposable by design, it is not backed up, and anything you want to keep should be rebuilt against a connection of your own.

What we are still working on

We would rather list this than imply it is finished.

  • Environment-scoped connections. QA and PROD are already first-class for variables, so one pipeline can read different schemas per environment. A connection, though, belongs to the organisation rather than to an environment. We want one that can be marked QA-only, so a pipeline still being built cannot reach production even by accident, and the production connection is attached deliberately at promotion. Of everything on this list, this is the one we most want to close.
  • Dedicated runners. Pipeline steps today execute on shared infrastructure, with separation between organisations enforced in software. For organisations whose own review will not accept that, a dedicated runner is something we will build — it is worth telling us early if that is you, because it changes what we build first.
  • The credentials screen is still sent stored secrets. When an Owner opens a connection to edit it, the stored values come down to the browser — as ciphertext now rather than plaintext. Only Owners reach that screen, and only for their own organisation, so this is hardening rather than exposure: the API should send a placeholder instead and treat an unchanged field on save as "keep what is stored".
  • No third-party security certification yet. If you need one to approve the tool, tell us which, because that is useful for deciding what to pursue first.

Security hardening here is continuous rather than finished, and this page gets updated when something changes — including when something on the list above comes off it.

The formal documents

Found something that worries you, or need an answer this page does not give? Book a demo and ask directly — a security question is a perfectly good reason to take one.