Column-Level Data Lineage: What It Is and Why Table-Level Isn’t Enough

Column-level data lineage maps every output column of a query, view, or report back to the exact source columns that produce it, together with each transformation applied along the way: functions, casts, joins, aggregations, and filters. Where table-level lineage says “table ordres feeds table revenue_report“, column-level lineage says “revenue_report.total est SOMME(commandes.montant), filtered by statut des commandes and grouped by clients.région“. It is the granularity at which impact analysis, debugging, and audit answers become precise instead of approximate.

This page defines the term, walks one output column through a real query with a CTE, a join, and an aggregate, and explains why extracting this information requires semantic SQL parsing rather than pattern matching. If lineage as a concept is new to you, start with what data lineage is and why it matters, then come back here for the column-level specifics.

What column-level data lineage records

For every column that a statement writes or returns, a column-level lineage graph stores three things:

• Source columns: the physical columns whose values flow into the output, resolved through any number of intermediate layers such as CTEs, subqueries, views, and temp tables.
• Transformations: the operations applied on the path, e.g. SOMME(), CAST(), CASE expressions, string functions, and set operators like UNION.
• Relationship type: whether a source column’s value actually lands in the output (direct lineage) or merely shapes the result through a filter, join, or grouping (indirect lineage). More on this distinction below, because most tools skip it.

Why table-level lineage isn’t enough

Table-level lineage is cheap to produce and genuinely useful for a first orientation: it draws the map of which tables and views feed which. The problem is that the questions data teams actually need answered are column questions, and table granularity forces you to over-approximate:

Question	Table-level answer	Column-level answer
What breaks if I rename clients.email?	Every object that reads clients — often dozens of false positives	Only the views, procedures, and reports that reference email specifically
Where does this wrong number in the dashboard come from?	“Somewhere upstream of these four tables”	The exact expression chain from the dashboard field back to physical source columns
Which outputs contain PII from ssn?	Every table downstream of the source table, whether or not ssn propagates	The precise set of columns the ssn value actually reaches
Can we drop this staging column?	Unknown — the table is referenced, so assume no	Yes, if no downstream column reads it directly or indirectly

The false-positive cost is not academic. If a schema change flags 40 downstream dashboards at table level but only 3 of them actually consume the changed column, the other 37 either get needlessly retested or, worse, teach the team to ignore the lineage tool. Precision is what makes lineage trustworthy, and precision lives at the column level.

A worked example: one column through a CTE, a join, and an aggregate

Here is a small but realistic query. The goal: trace exactly where total_revenue comes from.

WITH recent_orders AS (
    SELECT o.order_id,
           o.customer_id,
           o.amount,
           o.status
    FROM   orders o
    WHERE  o.order_date >= DATE '2026-01-01'
)
SELECT   c.region,
         SUM(r.amount) AS total_revenue
FROM     recent_orders r
JOIN     customers c ON c.customer_id = r.customer_id
WHERE    r.status = 'COMPLETED'
GROUP BY c.region;

Resolving total_revenue takes three hops:

1. total_revenue is defined as SUM(r.amount), so its value comes from column montant of the relation aliased r, passed through an aggregate function.
2. r is not a table. It is the CTE recent_orders, whose montant column is a pass-through of commandes.montant. The alias, the CTE, and the column projection all have to be resolved before the physical source is known.
3. The complete direct lineage is therefore a single physical column: commandes.montant → recent_orders.amount → SOMME() → total_revenue.

But five more columns shape the result without ever appearing in it: commandes.date_de_commande filters rows inside the CTE, statut des commandes filters them again in the outer OÙ, commandes.identifiant_client et customers.customer_id decide which rows join, and clients.région decides how rows group into aggregate buckets. Change the semantics of any of these and total_revenue changes, even though none of their values lands in the output. A text search would tell you “this query touches ordres et clients“. Column-level lineage tells you which one column carries the value and which five govern it.

Direct vs indirect lineage: the fdd/fddi distinction

Those two kinds of relationship deserve separate names. In Gudu SQLFlow’s lineage model, direct dataflow (labeled fdd) means a source column’s value flows into the target column, possibly transformed by functions, casts, or aggregates. Indirect dataflow (labeled fddi) means a source column influences the target without contributing its value: columns used in OÙ predicates, REJOINDRE conditions, GROUPER PAR keys, and similar clauses.

SQLFlow models these as distinct, separately toggleable relationship types in the diagram. Turn indirect lineage off to see pure value provenance, the view an auditor wants when proving which source fields end up in a regulated report. Turn it on to see the full impact surface, the view an engineer wants before touching a column, because dropping a filter column silently changes every downstream aggregate. Most competing lineage tools do not make this distinction at all: they either report only direct flow and miss real impact, or lump everything together and reintroduce the false positives you moved to column level to escape.

Why this requires semantic parsing, not regex

It is tempting to extract lineage by pattern-matching SQL text for table names and column names. That approach collapses on exactly the constructs production SQL is full of:

• Expansion stellaire. SÉLECTIONNER * names no columns at all. Which columns flow through depends on the schema of the underlying relation at analysis time, and in a join, on the combined schemas of all joined relations. Resolving * requires metadata plus scope rules; no text pattern can produce the column list.
• View resolution. When a query reads v_sales.net_amount, the true lineage runs through the view’s definition, and views routinely stack on other views. The analyzer must expand each definition recursively and splice the column mappings, an operation on a resolved semantic model, not on strings.
• Scope and ambiguity. An unqualified column like statut in a three-table join belongs to exactly one relation, determined by SQL’s scoping rules and the tables’ schemas. Guessing wrong silently corrupts the lineage graph.
• Expressions. COALESCE(a.x, b.y), CASE branches, and window functions each define a different set of contributing columns. Only a real expression tree captures them.
• Dialects. Oracle’s CONNECT BY, T-SQL’s OUTPUT clause, BigQuery’s EXCEPT in a star list: each dialect has syntax a generic ANSI grammar mis-parses. SQLFlow ships 39 dialect-specific parsers rather than one lowest-common-denominator grammar.

In short, accurate column-level lineage is a compiler problem. SQLFlow is built on the General SQL Parser, a full SQL compiler front-end (lexer, parser, semantic resolver, data-flow analyzer) developed commercially since the mid-2000s and validated against roughly 13,600 per-dialect SQL test fixtures. The same engine resolves the hardest cases: stored procedure bodies in Oracle PL/SQL and SQL Server T-SQL, lineage through procedure parameters and temp tables, and dynamic SQL assembled inside procedures.

Getting column-level lineage in practice

Gudu SQLFlow is an automated Outil de traçabilité des données SQL that produces the column-level graph described on this page from whatever SQL you have: pasted queries, uploaded files, live database metadata over JDBC, dbt manifest files, or Snowflake query history and Redshift query logs. The output is an interactive diagram you can drill into per column, plus structured lineage data as JSON or CSV, a PNG export, and a REST API.

At enterprise scale it batch-scans estates of 100+ databases and over a million columns, runs incremental scans against a persistent lineage repository, and exports to DataHub, Microsoft Purview, and OpenMetadata, so column-level lineage can live inside the catalog you already run. Because the analysis is static, SQLFlow reads SQL code and schema metadata only, never the rows in your tables, and the On-Premise edition keeps SQL text entirely inside your network.

For more traces like the one above, across subqueries, views, stored procedures, and dbt models, see our library of worked data lineage examples.

Foire aux questions

What is the difference between table-level and column-level data lineage?

Table-level lineage records which tables and views feed which, one edge per object pair. Column-level lineage records, for every output column, the exact source columns that feed it and the transformations applied. Column granularity is what removes false positives from impact analysis and makes audit answers exact.

What is indirect (impact) lineage?

Indirect lineage links a source column to an output it influences without contributing its value: columns in WHERE clauses, JOIN conditions, GROUP BY keys, and aggregate predicates. SQLFlow models it as a separate relationship type (fddi) that you can toggle independently of direct dataflow (fdd) in the diagram.

Can I extract column-level lineage with regex or grep?

Not reliably. Star expansion, view resolution, unqualified column names, and expressions like CASE or COALESCE all require a semantic model built from a real parse of the SQL plus schema metadata. Pattern matching can find table names; it cannot resolve which columns flow where.

Does column-level lineage work through views and SELECT *?

Yes, if the tool resolves them. SQLFlow expands SELECT * against schema metadata and recursively resolves column references through views, CTEs, and subqueries, so lineage runs from the final output column all the way to physical source columns.

Does it work on stored procedures and dynamic SQL?

Yes. SQLFlow has dedicated procedural parsers for Oracle PL/SQL and SQL Server T-SQL, traces lineage through procedure parameters and temp tables, resolves dynamic SQL assembled inside procedures, and renders a call graph of procedure-to-procedure invocations.

How do I get column-level lineage for my own SQL?

Paste a query into SQLFlow Cloud’s free tier and the column-level diagram renders in the browser. For whole databases, connect over JDBC or import a dbt manifest; for regulated environments, SQLFlow On-Premise runs in Docker or Kubernetes inside your own network.