SQLFlow REST API: Automate SQL Data Lineage Extraction

A data lineage API lets you extract lineage programmatically: you submit SQL code over HTTP and receive a structured lineage graph as JSON, listing every source table and column, every target, and the transformation relationships between them. The Gudu SQLFlow REST API does exactly this for 39 SQL dialects, at column-level granularity, including stored procedures and dynamic SQL. The same analysis that powers SQLFlow’s interactive diagrams is available as structured lineage data over HTTP.

Why put a data lineage API in your stack?

Interactive lineage diagrams are how humans explore lineage. But the highest-value uses of lineage are automated: blocking a pull request that silently breaks a downstream report, keeping a data catalog’s lineage current without manual curation, or answering “what feeds this column?” inside your own internal tooling. All of those need lineage as data, on demand, from a service you can call.

SQLFlow’s approach is static SQL analysis. The API parses the SQL text you send it and never touches the rows in your tables, which keeps the security review short: the only thing crossing the wire is SQL code, and with the on-premise edition even that stays inside your network. The engine underneath is the General SQL Parser, a commercial SQL compiler front-end developed since the mid-2000s and validated against roughly 13,600 per-dialect test fixtures, so the lineage you get back is built from a full semantic model of the SQL, not regex matching.

What the SQLFlow REST API returns

Conceptually, the API exposes the full SQLFlow analysis surface as JSON-over-HTTP:

Capability	You send	You get back
Lineage graph	SQL text plus a dialect identifier	The complete column-level lineage graph as JSON: tables, columns, and source-to-target relationships
Upstream / downstream trace	A table or column of interest	Only the subgraph feeding that object (upstream) or fed by it (downstream), for targeted impact analysis
Table-level lineage	SQL text, table granularity requested	A compact table-to-table dependency graph when column detail would be noise
Direct vs indirect lineage	A flag on the request	Pure dataflow relationships, or additionally the columns that shape results through WHERE, JOIN, and GROUP BY clauses
Stored procedure call graph	PL/SQL or T-SQL procedure bodies	Which procedures invoke which, with lineage traced through parameters, temp tables, and dynamic SQL
ER diagram	DDL scripts	Inferred primary/foreign-key relationships as an entity-relationship model
Exports	An analyzed job	JSON or CSV for machines, PNG for documentation and tickets

Every request names its dialect explicitly. SQLFlow ships dialect-specific parsers for 39 databases and query engines, from Snowflake, BigQuery, Databricks, and Redshift to Oracle, SQL Server, Teradata, Hive, and Trino, so vendor-specific syntax such as T-SQL temp tables is parsed as the vendor defines it, not approximated by a generic ANSI grammar. The full list is on the SQL data lineage tool overview.

A minimal example: POST SQL, receive a lineage graph

The core interaction is one HTTP call. You post SQL text and a dialect; the response is the lineage graph. Endpoint paths, authentication, and the complete parameter reference live in the SQLFlow API documentation; the shape of the exchange looks like this:

curl -s -X POST "$SQLFLOW_API/lineage" 
  -H "Authorization: Bearer $SQLFLOW_TOKEN" 
  --form "dbvendor=snowflake" 
  --form 'sqltext=CREATE VIEW customer_ltv AS
    SELECT c.customer_id,
           SUM(o.amount) AS lifetime_value
    FROM   customers c
    JOIN   orders o ON o.customer_id = c.customer_id
    WHERE  o.status = ''paid''
    GROUP  BY c.customer_id;'

The response is a graph: a list of database objects (tables, views, and their columns) and a list of relationships between columns. Trimmed and simplified for illustration:

{
  "relationships": [
    {
      "type": "direct",
      "target": {"object": "customer_ltv", "column": "lifetime_value"},
      "sources": [{"object": "orders", "column": "amount", "via": "SUM"}]
    },
    {
      "type": "direct",
      "target": {"object": "customer_ltv", "column": "customer_id"},
      "sources": [{"object": "customers", "column": "customer_id"}]
    },
    {
      "type": "indirect",
      "target": {"object": "customer_ltv", "column": "lifetime_value"},
      "sources": [{"object": "orders", "column": "status", "via": "WHERE"}]
    }
  ]
}

Note the third relationship. orders.status never appears in the view’s output, but it filters which rows are summed, so it absolutely affects lifetime_value. SQLFlow models this as indirect lineage, distinct from direct dataflow and requestable separately. Most lineage tools do not make this distinction, and it is precisely what an impact-analysis consumer needs: dropping orders.status would corrupt this view even though no output column “comes from” it.

Use case: feeding your data catalog

Catalog platforms are good at organizing metadata, ownership, and discovery; column-level SQL lineage is usually their weakest input. SQLFlow slots in as the lineage engine: analyze your SQL estate, then push the results into the catalog you already run. Enterprise deployments include ready-made export adapters for DataHub, Microsoft Purview, and OpenMetadata, and the JSON and CSV exports feed anything custom. At scale this runs as batch and incremental scans across estates of 100+ databases and over a million columns, with a persistent lineage repository, so the catalog stays current without anyone maintaining lineage by hand.

Use case: lineage inside your own tools

Internal data platforms grow the same features again and again: a “where does this metric come from?” panel, a deprecation checker, a migration dependency map. Rather than building a SQL parser, call the API from your service and render the graph however you like. If you want the interactive diagram itself rather than raw JSON, SQLFlow also ships an embeddable JavaScript widget with a 30+ method API that drops into any web app; for backend-only JVM integration there is a Java library exposing the same engine. API, widget, and library all return consistent results because they share one parser.

How does this compare to other programmatic lineage options?

This is a SQL analysis service, not a parser library or a runtime collector. Three categories are worth knowing. Open-source parsing libraries like sqllineage and sqlglot are genuinely good for parsing individual statements inside a Python process, and for straightforward SELECT and INSERT logic they may be enough; the gap appears on stored procedures, dynamic SQL, dialect edge cases, and view or star-expansion resolution that needs schema context. Runtime lineage standards like OpenLineage capture lineage from job execution events, which is excellent for orchestrator-level, run-by-run lineage but only sees code that actually ran, at the granularity the emitting integration provides. Catalog-first platforms expose lineage APIs over whatever lineage they ingested, so their answers are only as deep as the ingestion. SQLFlow’s REST API is a different tool: a specialized SQL analysis service that computes column-level lineage from the code itself, including the SQL that never ran and the procedures other tools skip. The categories combine well; many teams use SQLFlow to compute lineage and a catalog to serve it to end users.

Where the API runs: cloud or your own network

The REST API is available in both SQLFlow deployments. SQLFlow Cloud is the fastest start: a SaaS with a free tier, premium at $49.99/month, no infrastructure to run. SQLFlow On-Premise deploys via Docker or Kubernetes inside your network, works fully air-gapped, and is priced at $500/month or $4,800 one-time per selected database type, installable on two servers. A common pattern is to prototype the integration against Cloud, then move the same API calls to an on-premise host so SQL text never leaves your infrastructure.

Frequently asked questions

Does the API return column-level or table-level lineage?

Both. Column-level is the default depth: for each output column you get the exact source columns and the functions, joins, and set operations between them, with direct and indirect lineage distinguished. You can request table-level granularity when a compact dependency graph is all you need.

Which SQL dialects does the data lineage API support?

39 dialects, each with its own parser: Snowflake, BigQuery, Redshift, Databricks, Oracle, SQL Server, PostgreSQL, MySQL, Teradata, Hive, Spark SQL, Trino, and more. You name the dialect in each request.

Can it analyze stored procedures and dynamic SQL?

Yes. Oracle PL/SQL and SQL Server T-SQL have dedicated procedural parsers. Lineage is traced through procedure parameters and temp tables, dynamic SQL built inside procedures is resolved and analyzed, and the API can return the procedure-to-procedure call graph.

Does the API need access to my data?

No. It performs static analysis of the SQL text you submit and never reads table row data. With the on-premise edition, the SQL text itself also stays inside your network.

What output formats are available?

JSON is the primary format for programmatic use; CSV export suits spreadsheets and bulk loads, and PNG export produces the rendered diagram for documentation. Enterprise deployments add export adapters for DataHub, Microsoft Purview, and OpenMetadata.

Where is the full API reference?

At docs.gudusoft.com, covering endpoint paths, authentication, request parameters per capability, and the complete response schema. For which plans include API access, see SQLFlow pricing.