Orchestrate data movement without the glue code
Design, schedule, and monitor ELT and ETL pipelines visually — from extraction through transformation to your warehouse — with durable runs, retries, alerting, and end-to-end lineage into your dashboards.
- Visual designer — build pipelines through the UI, no glue code
- Incremental loads and change data capture
- Durable runs resume after failure — with lineage into your BI
See it run
One canvas, source to dashboard
Extract, transform, quality-check, and load — orchestrated as one durable run, with lineage that reaches all the way to the dashboard.
Always in view
Every run, logged and recoverable
Row counts, durations, and errors on every run — retry from the failed step, backfill a date range, and watch volumes trend over time.
Rows loaded · per run
A closer look
Inside Pipelines
Build pipelines, not glue code
Compose extract → transform → load as a visual graph through the UI or API — versioned and run for you. No orchestration servers to operate, no Python files to babysit.
- Visual DAG with copy, transform, branch, and loop steps
- Cron, file-arrival, API, and dependency triggers
- Backfill any date range as parameterized runs
Durable runs that resume
A crashed run picks up mid-pipeline without repeating completed work. Durable execution, per-step retries with backoff, and checkpoint restarts keep long loads resilient.
- Resume mid-pipeline after a worker restart
- Per-step retry policy with exponential backoff
- Atomic partition swap — dashboards never see a half-load
Lineage into your dashboards
Column-level lineage flows automatically from source through every transform into the dashboards built on top — so impact analysis and trust come for free, inside one platform.
- Source → dataset → transform → dashboard card
- One login and audit trail across ingest to insight
- Incremental and change data capture keep syncs cheap
What you can do with Pipelines
Visual designer
Compose pipelines as a graph of activities — copy, transform, branch, and loop — built through the UI or API, versioned, and run for you.
Flexible triggers
Run on a cron schedule, on file arrival, or on demand via API. Chain pipelines with dependency triggers and backfill any date range.
Incremental & CDC
Move only what changed. Watermark-based incremental loads and change data capture keep syncs fast and cheap.
Transforms you control
Transform with SQL models that run right in your warehouse, or visual dataflow steps — joins, filters, aggregations, and type casting.
Durable, resumable runs
A crashed run resumes mid-pipeline without re-doing completed work. Per-step retries with backoff and checkpoint restarts keep long loads resilient.
Monitoring & lineage
Every run logs rows, durations, and errors, with alerts on failures. Column-level lineage flows from source through transforms into every dashboard.
Durable
Runs resume after failure
Batch · CDC
Incremental change capture
End-to-end
Source-to-dashboard lineage
How it works
Pick source & destination
Any connector to your warehouse or lake
Design the flow
Add transform, branch, and validation steps
Set the trigger
Schedule, event, or API-driven runs
Monitor runs
Track history, alerts, and data volumes
Frequently asked questions
Pipelines covers the core ADF workloads — copy activities, transforms, triggers, and monitoring — inside the same platform as your BI. Lineage flows end to end and metering is in plain execution-hours, with no separate tools to stitch together.