Welcome to data-conductor

data-conductor is a comprehensive data pipeline and workflow platform that enables you to build, deploy, and manage data workflows with ease. Whether you're processing SQL data, running automated pipelines, or exposing workflows as APIs, data-conductor provides the tools you need.

data-conductor Dashboard Overview Main dashboard showing your workflows and deployments

What is data-conductor?

data-conductor is designed for data teams, developers, and organizations who need to:

  • Build Data Pipelines: Create SQL-based data steps that connect to your databases
  • Automate Workflows: Schedule pipelines to run automatically using CRON expressions
  • Execute On-Demand: Trigger pipeline executions immediately when needed
  • Expose APIs: Turn your workflows into secure API endpoints
  • Manage at Scale: Handle multiple environments, versions, and deployments

Key Features

🔧 Data Builder

Create and manage SQL data steps that connect to your databases. Build complex data transformations with an intuitive SQL editor.

Learn more about Data Builder →

🚀 Deployment Manager

Deploy your workflows as CRON jobs, immediate executions, or API endpoints. Monitor job executions and manage deployments.

Learn more about Deployment Manager →

🏢 Organization Management

Configure your organization settings, manage API tokens, set up trusted IP addresses, and control access.

Learn more about Organization Settings →

🔒 Security Features

  • IP-based access control with CIDR support
  • API token authentication
  • Audit logging for security events
  • Role-based access control

Quick Start

Ready to get started? Follow our quick setup guide:

  1. Set up your organization
  2. Create your first data step
  3. Deploy your pipeline

Video Overview

data-conductor Platform Overview 5-minute overview of data-conductor's main features

Platform Architecture

data-conductor is built with modern technologies to ensure scalability and reliability:

  • Frontend: Next.js 15 with React 19 and TypeScript
  • Backend: Node.js with Express and PostgreSQL
  • Security: JWT authentication with comprehensive audit logging
  • Deployment: Support for Google Cloud and containerized environments

Getting Help

Need assistance? Here are your options:

  • Troubleshooting Guide - Common issues and solutions
  • FAQ - Frequently asked questions
  • Tutorials - Step-by-step guides for common tasks

What's Next?

Choose your learning path:

Let's build something amazing with your data! 🚀