{"id":6761,"date":"2026-07-12T02:07:41","date_gmt":"2026-07-12T10:07:41","guid":{"rendered":"https:\/\/www.gudusoft.com\/spark-sql-data-lineage\/"},"modified":"2026-07-12T02:07:41","modified_gmt":"2026-07-12T10:07:41","slug":"spark-sql-data-lineage","status":"publish","type":"page","link":"https:\/\/www.gudusoft.com\/jp\/spark-sql-data-lineage\/","title":{"rendered":"Spark SQL\u30c7\u30fc\u30bf\u30ea\u30cd\u30fc\u30b8\uff1aSpark\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u306e\u5217\u30ea\u30cd\u30fc\u30b8"},"content":{"rendered":"<p><strong>Spark SQL\u30c7\u30fc\u30bf\u30ea\u30cd\u30fc\u30b8<\/strong> \u3053\u308c\u306f\u3001\u30c7\u30fc\u30bf\u304c Spark SQL \u30b3\u30fc\u30c9\u5185\u3092\u3069\u306e\u3088\u3046\u306b\u79fb\u52d5\u3059\u308b\u304b\u3092\u793a\u3059\u5217\u30ec\u30d9\u30eb\u306e\u30de\u30c3\u30d7\u3067\u3059\u3002\u3069\u306e\u30bd\u30fc\u30b9\u5217\u304c\u3001\u306b\u3088\u3063\u3066\u66f8\u304d\u8fbc\u307e\u308c\u308b\u3059\u3079\u3066\u306e\u30c6\u30fc\u30d6\u30eb\u306b\u30c7\u30fc\u30bf\u3092\u4f9b\u7d66\u3057\u3066\u3044\u308b\u304b\u3092\u793a\u3057\u307e\u3059\u3002 <code>\u30c6\u30fc\u30d6\u30eb\u3092\u4f5c\u6210\u3057\u307e\u3059...AS SELECT<\/code> \u307e\u305f <code>\u633f\u5165\u4e0a\u66f8\u304d<\/code>\u305d\u3057\u3066\u3001\u305d\u306e\u904e\u7a0b\u3067\u4e00\u6642\u7684\u306a\u30d3\u30e5\u30fc\u3001\u7d50\u5408\u3001\u95a2\u6570\u3001\u30d5\u30a3\u30eb\u30bf\u30fc\u304c\u5b9f\u884c\u3055\u308c\u307e\u3059\u3002 <strong>Gudu SQLFlow<\/strong> \u5c02\u7528\u306eSpark\u65b9\u8a00\u30d1\u30fc\u30b5\u30fc\u3092\u4f7f\u7528\u3057\u3066Spark SQL\u3092\u9759\u7684\u306b\u89e3\u6790\u3059\u308b\u3053\u3068\u3067\u3001\u305d\u306e\u30de\u30c3\u30d7\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u30af\u30e9\u30b9\u30bf\u30fc\u306e\u8a08\u6e2c\u3001\u30ea\u30b9\u30ca\u30fc\u306e\u30a2\u30bf\u30c3\u30c1\u3001\u30b8\u30e7\u30d6\u306e\u5b9f\u884c\u306f\u4e0d\u8981\u3067\u3059\u3002SQL\u3092\u8cbc\u308a\u4ed8\u3051\u308b\u3060\u3051\u3067\u3001\u7cfb\u7d71\u56f3\u3092\u53d6\u5f97\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<div class=\"wp-block-group alignfull has-background is-layout-constrained\" style=\"background-color:#eef7fb;padding-top:24px;padding-bottom:24px\"><div class=\"wp-block-group__inner-container\">\n\n<p><strong>\u4eca\u3059\u3050\u304a\u8a66\u3057\u304f\u3060\u3055\u3044\uff1a<\/strong> Spark SQL \u30b9\u30af\u30ea\u30d7\u30c8\u3092\u8cbc\u308a\u4ed8\u3051\u308b <a href=\"https:\/\/sqlflow.gudusoft.com\/?utm_source=gudusoft&amp;utm_medium=website&amp;utm_campaign=spark-sql-data-lineage\" target=\"_blank\" rel=\"noreferrer noopener\">\u7121\u6599\u30aa\u30f3\u30e9\u30a4\u30f3SQL\u7cfb\u7d71\u53ef\u8996\u5316\u30c4\u30fc\u30eb<\/a>Spark\u65b9\u8a00\u3092\u9078\u629e\u3057\u3001\u4efb\u610f\u306e\u5217\u3092\u4e0a\u6d41\u307e\u305f\u306f\u4e0b\u6d41\u306b\u30c8\u30ec\u30fc\u30b9\u3057\u307e\u3059\u3002\u30af\u30e9\u30a6\u30c9\u7248\u306b\u306f\u7121\u6599\u67a0\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n\n<\/div><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">\u30e9\u30f3\u30bf\u30a4\u30e0\u30ea\u30b9\u30ca\u30fc\u3068\u9759\u7684\u89e3\u6790\uff1aSpark\u306e\u5c65\u6b74\u3092\u53d6\u5f97\u3059\u308b2\u3064\u306e\u65b9\u6cd5<\/h2>\n\n\n\n<p>Spark\u306e\u4e16\u754c\u306b\u304a\u3051\u308b\u7cfb\u8b5c\u7ba1\u7406\u306b\u306f\u6839\u672c\u7684\u306b\u7570\u306a\u308b2\u3064\u306e\u30a2\u30d7\u30ed\u30fc\u30c1\u304c\u3042\u308a\u3001\u307b\u3068\u3093\u3069\u306e\u30c1\u30fc\u30e0\u306f\u6700\u7d42\u7684\u306b\u4e21\u65b9\u3092\u5fc5\u8981\u3068\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>\u30e9\u30f3\u30bf\u30a4\u30e0\u306e\u7cfb\u8b5c<\/strong> OpenLineage\u3068\u305d\u306eSpark\u7d71\u5408\u304c\u63a1\u7528\u3057\u3066\u3044\u308b\u30a2\u30d7\u30ed\u30fc\u30c1\u306f\u3001Spark\u30bb\u30c3\u30b7\u30e7\u30f3\u306b\u30ea\u30b9\u30ca\u30fc\u3092\u30a2\u30bf\u30c3\u30c1\u3057\u3001\u30b8\u30e7\u30d6\u306e\u5b9f\u884c\u3054\u3068\u306b\u30ea\u30cd\u30fc\u30b8\u30a4\u30d9\u30f3\u30c8\u3092\u767a\u884c\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u975e\u5e38\u306b\u512a\u308c\u305f\u6a5f\u80fd\u3067\u3001\u9759\u7684\u89e3\u6790\u3067\u306f\u628a\u63e1\u3067\u304d\u306a\u3044\u5b9f\u884c\u30ec\u30d9\u30eb\u306e\u30e1\u30bf\u30c7\u30fc\u30bf\u3068\u3068\u3082\u306b\u3001\u5b9f\u969b\u306b\u4f55\u304c\u3001\u3044\u3064\u3001\u3069\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u5bfe\u3057\u3066\u5b9f\u884c\u3055\u308c\u305f\u304b\u3092\u8a18\u9332\u3057\u307e\u3059\u3002\u300c\u6628\u6669\u306e\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u306f\u5b9f\u969b\u306b\u4f55\u306b\u5f71\u97ff\u3092\u4e0e\u3048\u305f\u306e\u304b\uff1f\u300d\u3068\u3044\u3046\u7591\u554f\u306b\u3001\u30e9\u30f3\u30bf\u30a4\u30e0\u30ea\u30cd\u30fc\u30b8\u304c\u7b54\u3048\u3066\u304f\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>\u9759\u7684\u7cfb\u7d71<\/strong> SQLFlow\u306e\u30a2\u30d7\u30ed\u30fc\u30c1\u306f\u3001\u30b3\u30f3\u30d1\u30a4\u30e9\u304c\u884c\u3046\u3088\u3046\u306b\u3001SQL\u30c6\u30ad\u30b9\u30c8\u81ea\u4f53\u3092\u89e3\u6790\u3057\u3001\u30b3\u30fc\u30c9\u304b\u3089\u7cfb\u7d71\u60c5\u5831\u3092\u5c0e\u304d\u51fa\u3057\u307e\u3059\u3002\u3053\u308c\u306b\u3088\u308a\u3001\u30e9\u30f3\u30bf\u30a4\u30e0\u30ad\u30e3\u30d7\u30c1\u30e3\u3067\u306f\u69cb\u9020\u7684\u306b\u51e6\u7406\u3067\u304d\u306a\u3044\u3059\u3079\u3066\u306e\u3053\u3068\u304c\u7db2\u7f85\u3055\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<ul><li><strong>\u307e\u3060\u5b9f\u884c\u3057\u3066\u3044\u306a\u3044SQL\u3002<\/strong> \u30d7\u30eb\u30ea\u30af\u30a8\u30b9\u30c8\u306b\u542b\u307e\u308c\u308b\u65b0\u3057\u3044\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u3001\u5225\u306e\u30c1\u30fc\u30e0\u304b\u3089\u5f15\u304d\u7d99\u3044\u3060\u30b9\u30af\u30ea\u30d7\u30c8\u3001\u56db\u534a\u671f\u3054\u3068\u306b\u30b9\u30b1\u30b8\u30e5\u30fc\u30eb\u3055\u308c\u305f\u30b8\u30e7\u30d6\u306e\u30d0\u30c3\u30af\u30ed\u30b0\u306a\u3069\u3001\u3053\u308c\u3089\u306f\u5b9f\u884c\u3055\u308c\u308b\u307e\u3067\uff08\u5834\u5408\u306b\u3088\u3063\u3066\u306f\u672c\u756a\u30c7\u30fc\u30bf\u306b\u5bfe\u3057\u3066\uff09\u30e9\u30f3\u30bf\u30a4\u30e0\u30a4\u30d9\u30f3\u30c8\u3092\u767a\u884c\u3057\u307e\u305b\u3093\u3002<\/li>\n<li><strong>\u30b3\u30fc\u30c9\u30ec\u30d3\u30e5\u30fc\u3002<\/strong> \u30ec\u30d3\u30e5\u30fc\u62c5\u5f53\u8005\u306f\u3001\u5909\u66f4\u3092\u627f\u8a8d\u3059\u308b\u524d\u306b\u3001\u5909\u66f4\u306e\u5217\u30ec\u30d9\u30eb\u306e\u4f9d\u5b58\u95a2\u4fc2\u30b0\u30e9\u30d5\u5168\u4f53\u3092\u78ba\u8a8d\u3067\u304d\u308b\u305f\u3081\u3001\u30c7\u30d7\u30ed\u30a4\u5f8c\u306b\u305d\u306e\u5f71\u97ff\u3092\u767a\u898b\u3059\u308b\u5fc5\u8981\u304c\u306a\u304f\u306a\u308a\u307e\u3059\u3002<\/li>\n<li><strong>\u79fb\u884c\u76e3\u67fb\u3002<\/strong> \u30ef\u30fc\u30af\u30ed\u30fc\u30c9\u3092 Spark \u306b\u79fb\u884c\u3059\u308b\u5834\u5408 (\u901a\u5e38\u306f Hive \u304b\u3089)\u3001\u4f9d\u5b58\u95a2\u4fc2\u30b0\u30e9\u30d5\u304c\u5fc5\u8981\u306b\u306a\u308a\u307e\u3059\u3002 <em>\u5168\u4f53<\/em> \u6708\u672b\u306b\u306e\u307f\u5b9f\u884c\u3055\u308c\u308b\u30b8\u30e7\u30d6\u3092\u542b\u3080\u30b3\u30fc\u30c9\u30d9\u30fc\u30b9\u3002\u9759\u7684\u89e3\u6790\u3067\u306f\u3001\u3059\u3079\u3066\u3092\u4e00\u5ea6\u306b\u8aad\u307f\u8fbc\u3080\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-table\"><table><thead><tr><th><\/th><th>\u30e9\u30f3\u30bf\u30a4\u30e0\u30ea\u30b9\u30ca\u30fc (OpenLineage)<\/th><th>\u9759\u7684\u89e3\u6790\uff08SQLFlow\uff09<\/th><\/tr><\/thead><tbody>\n<tr><td>\u8840\u7d71\u304c\u6349\u3048\u3089\u308c\u305f<\/td><td>\u30b8\u30e7\u30d6\u5b9f\u884c\u3054\u3068\u306b\u3001\u5b9f\u884c\u3055\u308c\u308b<\/td><td>SQL\u30c6\u30ad\u30b9\u30c8\u304b\u3089\u3001\u4f55\u304b\u304c\u5b9f\u884c\u3055\u308c\u308b\u524d\u306b<\/td><\/tr>\n<tr><td>\u5fc5\u8981<\/td><td>Spark\u30bb\u30c3\u30b7\u30e7\u30f3\u3067\u30ea\u30b9\u30ca\u30fc\u304c\u8a2d\u5b9a\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u30b8\u30e7\u30d6\u306f\u5b9f\u884c\u3055\u308c\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002<\/td><td>SQL\u30d5\u30a1\u30a4\u30eb\u3068\u3001\u30aa\u30d7\u30b7\u30e7\u30f3\u306e\u30b9\u30ad\u30fc\u30de\u30e1\u30bf\u30c7\u30fc\u30bf<\/td><\/tr>\n<tr><td>\u672a\u5b9f\u884c\u306e\u30b3\u30fc\u30c9\u3092\u30ab\u30d0\u30fc\u3057\u307e\u3059<\/td><td>\u3044\u3044\u3048\u3001\u30e9\u30f3\u30cb\u30f3\u30b0\u3082\u30a4\u30d9\u30f3\u30c8\u3082\u3042\u308a\u307e\u305b\u3093\u3002<\/td><td>\u306f\u3044\u3001PR\u3001\u30d0\u30c3\u30af\u30ed\u30b0\u3001\u3081\u3063\u305f\u306b\u5b9f\u884c\u3055\u308c\u306a\u3044\u30b8\u30e7\u30d6<\/td><\/tr>\n<tr><td>\u5b9f\u884c\u30e1\u30bf\u30c7\u30fc\u30bf\uff08\u30bf\u30a4\u30df\u30f3\u30b0\u3001\u30d0\u30fc\u30b8\u30e7\u30f3\uff09<\/td><td>\u306f\u3044<\/td><td>\u3044\u3044\u3048 \u2014 \u30b3\u30fc\u30c9\u30ec\u30d9\u30eb\u306e\u307f<\/td><\/tr>\n<tr><td>\u5178\u578b\u7684\u306a\u4f7f\u7528\u4f8b<\/td><td>\u7a3c\u50cd\u4e2d\u306e\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u306e\u904b\u7528\u4e0a\u306e\u53ef\u89b3\u6e2c\u6027<\/td><td>\u5f71\u97ff\u5206\u6790\u3001\u30b3\u30fc\u30c9\u30ec\u30d3\u30e5\u30fc\u3001\u79fb\u884c\u76e3\u67fb\u3001\u30b3\u30f3\u30d7\u30e9\u30a4\u30a2\u30f3\u30b9\u6587\u66f8\u4f5c\u6210<\/td><\/tr>\n<\/tbody><\/table><\/figure>\n\n\n\n<p>\u3053\u308c\u3089\u306f\u4e92\u3044\u306b\u88dc\u5b8c\u3057\u5408\u3046\u3082\u306e\u3067\u3042\u308a\u3001\u7af6\u5408\u3059\u308b\u3082\u306e\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\u30e9\u30f3\u30bf\u30a4\u30e0\u30ea\u30cd\u30fc\u30b8\u306f\u767a\u751f\u3057\u305f\u4e8b\u8c61\u3092\u793a\u3057\u3001\u9759\u7684\u30ea\u30cd\u30fc\u30b8\u306f\u30b3\u30fc\u30c9\u306e\u52d5\u4f5c\u3092\u793a\u3057\u307e\u3059\u3002SQLFlow \u306e DataHub\u3001Microsoft Purview\u3001\u304a\u3088\u3073 OpenMetadata \u7528\u306e\u30a8\u30af\u30b9\u30dd\u30fc\u30c8\u30a2\u30c0\u30d7\u30bf\u3092\u4f7f\u7528\u3059\u308b\u3068\u3001\u9759\u7684\u306b\u751f\u6210\u3055\u308c\u305f\u30ea\u30cd\u30fc\u30b8\u3092\u30e9\u30f3\u30bf\u30a4\u30e0\u30a4\u30d9\u30f3\u30c8\u3068\u540c\u3058\u30ab\u30bf\u30ed\u30b0\u306b\u683c\u7d0d\u3067\u304d\u308b\u305f\u3081\u30012 \u3064\u306e\u30d3\u30e5\u30fc\u3092\u4e26\u3079\u3066\u8868\u793a\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u4f8b\uff1aCTAS\u3078\u306e\u4e00\u6642\u30d3\u30e5\u30fc\u30c1\u30a7\u30fc\u30f3<\/h2>\n\n\n\n<p>Spark SQL \u3067\u5358\u7d14\u306a\u30ea\u30cd\u30fc\u30b8\u30c4\u30fc\u30eb\u3092\u7121\u52b9\u5316\u3059\u308b\u30d1\u30bf\u30fc\u30f3\u306f\u3001\u4e00\u6642\u30d3\u30e5\u30fc\u30c1\u30a7\u30fc\u30f3\u3067\u3059\u3002\u30ce\u30fc\u30c8\u30d6\u30c3\u30af\u3068\u30b8\u30e7\u30d6\u306f\u3001\u5b9a\u671f\u7684\u306b\u30ed\u30b8\u30c3\u30af\u3092\u30b9\u30c6\u30fc\u30b8\u30f3\u30b0\u3057\u307e\u3059\u3002 <code>\u4e00\u6642\u30d3\u30e5\u30fc\u306e\u4f5c\u6210\u307e\u305f\u306f\u7f6e\u63db<\/code> \u6700\u7d42\u6bb5\u968e\u307e\u3067\u306e\u30b9\u30c6\u30c3\u30d7 <code>\u30c6\u30fc\u30d6\u30eb\u3092\u4f5c\u6210\u3057\u307e\u3059...AS SELECT<\/code>\u4ee5\u4e0b\u306b\u3001\u7c21\u6f54\u306a\u304c\u3089\u3082\u73fe\u5b9f\u7684\u306a\u4f8b\u3092\u793a\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>CREATE OR REPLACE TEMP VIEW clean_events AS SELECT CAST(event_time AS TIMESTAMP) AS event_ts, user_id, lower(event_name) AS event_name, amount FROM raw.events WHERE event_date &gt;= date_sub(current_date(), 30); CREATE OR REPLACE TEMP VIEW user_daily AS SELECT user_id, to_date(event_ts) AS event_day, SUM(amount) AS daily_spend FROM clean_events WHERE event_name = &#039;purchase&#039; GROUP BY user_id, to_date(event_ts); CREATE TABLE analytics.user_spend_30d USING PARQUET AS SELECT u.user_id, u.segment, d.event_day, d.daily_spend FROM user_daily d JOIN dim.users u ON u.user_id = d.user_id;<\/code><\/pre>\n\n\n\n<p>\u300c\u3069\u3053\u3067 <code>\u30a2\u30ca\u30ea\u30c6\u30a3\u30af\u30b9.user_spend_30d.daily_spend<\/code> \u300c\u3069\u3053\u304b\u3089\u6765\u305f\u306e\u304b\uff1f\u300d\u3068\u3044\u3046\u7591\u554f\u306b\u5bfe\u3059\u308b\u6b63\u76f4\u306a\u7b54\u3048\u306f\u30012\u3064\u306e\u30d3\u30e5\u30fc\u30ec\u30a4\u30e4\u30fc\u3092\u89e3\u6c7a\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b\u3002 <code>\u6bce\u65e5\u306e\u652f\u51fa<\/code> \u306f <code>SUM(clean_events.amount)<\/code>\u3001 \u3068 <code>clean_events.amount<\/code> \u306f <code>\u751f\u306e\u30a4\u30d9\u30f3\u30c8\u6570<\/code>SQLFlow \u306f\u307e\u3055\u306b\u305d\u306e\u89e3\u6c7a\u3092\u5b9f\u884c\u3057\u3001\u5b8c\u5168\u306a\u30c1\u30a7\u30fc\u30f3\u3092\u30ec\u30f3\u30c0\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002 <code>\u751f\u306e\u30a4\u30d9\u30f3\u30c8\u6570<\/code> \u3092\u901a\u3057\u3066 <code>\u30af\u30ea\u30fc\u30f3\u30a4\u30d9\u30f3\u30c8<\/code> \u3068 <code>\u30e6\u30fc\u30b6\u30fc\u30c7\u30a4\u30ea\u30fc<\/code> \u6700\u7d42\u30c6\u30fc\u30d6\u30eb\u306b\u9032\u51fa\u3057\u3001 <code>\u548c<\/code> \u96c6\u7d04\u306f\u3001\u305d\u308c\u304c\u8d77\u3053\u308b\u30a8\u30c3\u30b8\u306b\u4ed8\u968f\u3059\u308b\u3002<\/p>\n\n\n\n<p>\u307e\u305f\u3001\u307b\u3068\u3093\u3069\u306e\u30c4\u30fc\u30eb\u304c\u5b8c\u5168\u306b\u7121\u8996\u3057\u3066\u3044\u308b\u60c5\u5831\u3082\u6349\u3048\u3066\u3044\u307e\u3059\u3002 <strong>\u9593\u63a5\u7684\u306a\u7cfb\u7d71<\/strong>\u5217 <code>\u30a4\u30d9\u30f3\u30c8\u65e5\u4ed8<\/code>, <code>\u30a4\u30d9\u30f3\u30c8\u540d<\/code>\u3001\u305d\u3057\u3066\u7d50\u5408\u30ad\u30fc <code>\u30e6\u30fc\u30b6\u30fcID<\/code> \u6c7a\u3057\u3066\u767b\u5834\u3057\u306a\u3044 <code>\u6bce\u65e5\u306e\u652f\u51fa<\/code>\u3057\u304b\u3057\u30013\u3064\u3059\u3079\u3066\u304c\u305d\u306e\u4fa1\u5024\u3092\u5de6\u53f3\u3059\u308b\u300230\u65e5\u9593\u30d5\u30a3\u30eb\u30bf\u30fc\u3092\u5909\u66f4\u3059\u308b\u304b\u3001 <code>&#039;\u8cfc\u5165&#039;<\/code> \u8ff0\u8a9e\u3068\u51fa\u529b\u306e\u3059\u3079\u3066\u306e\u6570\u5024\u304c\u5909\u66f4\u3055\u308c\u307e\u3059\u3002SQLFlow \u306f\u3053\u308c\u3089\u3092\u500b\u5225\u306e\u5207\u308a\u66ff\u3048\u53ef\u80fd\u306a\u95a2\u4fc2\u30bf\u30a4\u30d7\u3068\u3057\u3066\u30e2\u30c7\u30eb\u5316\u3059\u308b\u305f\u3081\u3001\u7d14\u7c8b\u306a\u30c7\u30fc\u30bf\u30d5\u30ed\u30fc\u3092\u8868\u793a\u3059\u308b\u3053\u3068\u3082\u3001\u5f71\u97ff\u7bc4\u56f2\u5168\u4f53\u3092\u8868\u793a\u3059\u308b\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002\u30b9\u30ad\u30fc\u30de\u5909\u66f4\u306e\u30ec\u30d3\u30e5\u30fc\u3067\u306f\u3001\u5f71\u97ff\u7bc4\u56f2\u3092\u8868\u793a\u3059\u308b\u30d3\u30e5\u30fc\u304c\u5b9f\u969b\u306b\u5fc5\u8981\u306a\u30d3\u30e5\u30fc\u3067\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Spark SQL \u306f SQLFlow \u3092\u89e3\u6c7a\u3057\u307e\u3059<\/h2>\n\n\n\n<p>SQLFlow\u306b\u306f\u3001Spark SQL\u65b9\u8a00\u30d1\u30fc\u30b5\u30fc\u304c\u540c\u68b1\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u6c4e\u7528\u7684\u306aANSI\u6587\u6cd5\u306bSpark\u30ad\u30fc\u30ef\u30fc\u30c9\u3092\u8ffd\u52a0\u3057\u305f\u3082\u306e\u3067\u306f\u306a\u304f\u300139\u7a2e\u985e\u3042\u308b\u65b9\u8a00\u56fa\u6709\u306e\u30d1\u30fc\u30b5\u30fc\u306e1\u3064\u3067\u3059\u3002\u7279\u306bSpark SQL\u3067\u306f\u3001\u4ee5\u4e0b\u306e\u5c65\u6b74\u3092\u6301\u3064\u30b9\u30c6\u30fc\u30c8\u30e1\u30f3\u30c8\u3092\u51e6\u7406\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<ul><li><strong><code>\u30c6\u30fc\u30d6\u30eb\u3092\u4f5c\u6210\u3057\u307e\u3059...AS SELECT<\/code> \uff08CTAS\uff09<\/strong> \u2014 Spark \u30d0\u30c3\u30c1 \u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u306e\u4e3b\u529b\u6a5f\u80fd\u3002\u3059\u3079\u3066\u306e\u51fa\u529b\u5217\u306f\u3001\u5b8c\u5168\u306a SELECT \u3092\u901a\u3058\u3066\u305d\u306e\u30bd\u30fc\u30b9\u307e\u3067\u8ffd\u8de1\u3055\u308c\u307e\u3059\u3002<\/li>\n<li><strong><code>\u633f\u5165\u5148<\/code> \u3068 <code>\u633f\u5165\u4e0a\u66f8\u304d<\/code><\/strong> \u2014 SELECT\u30ea\u30b9\u30c8\u3068\u5bfe\u8c61\u30c6\u30fc\u30d6\u30eb\u306e\u30b9\u30ad\u30fc\u30de\u9593\u306e\u4f4d\u7f6e\u7684\u306a\u5217\u30de\u30c3\u30d4\u30f3\u30b0\u3092\u542b\u307f\u307e\u3059\u3002<\/li>\n<li><strong><code>\u4e00\u6642\u30d3\u30e5\u30fc\u306e\u4f5c\u6210\u307e\u305f\u306f\u7f6e\u63db<\/code><\/strong> \u2015 \u4e00\u6642\u30d3\u30e5\u30fc\u306f\u89e3\u6c7a\u3055\u308c\u3001\u9023\u9396\u3055\u308c\u308b\u305f\u3081\u3001\u30ea\u30cd\u30fc\u30b8\u306f\u30b9\u30c6\u30fc\u30b8\u30f3\u30b0\u30ec\u30a4\u30e4\u30fc\u3067\u505c\u6b62\u3059\u308b\u306e\u3067\u306f\u306a\u304f\u3001\u30b9\u30c6\u30fc\u30b8\u30f3\u30b0\u30ec\u30a4\u30e4\u30fc\u3092\u901a\u904e\u3057\u307e\u3059\u3002<\/li>\n<li><strong>CTE\u3001\u30b5\u30d6\u30af\u30a8\u30ea\u3001 <code>\u9078\u629e *<\/code><\/strong> \u5217\u53c2\u7167\u306f\u5171\u901a\u30c6\u30fc\u30d6\u30eb\u5f0f\u3068\u30cd\u30b9\u30c8\u3055\u308c\u305f\u30b5\u30d6\u30af\u30a8\u30ea\u306b\u3088\u3063\u3066\u89e3\u6c7a\u3055\u308c\u3001\u30b9\u30ad\u30fc\u30de\u30e1\u30bf\u30c7\u30fc\u30bf\u304c\u5229\u7528\u53ef\u80fd\u306a\u5834\u5408\u306f\u30b9\u30bf\u30fc\u5f0f\u304c\u5b9f\u969b\u306e\u5217\u306b\u5c55\u958b\u3055\u308c\u307e\u3059\u3002<\/li>\n<li><strong>\u95a2\u6570\u3001\u578b\u5909\u63db\u3001\u304a\u3088\u3073\u96c6\u5408\u6f14\u7b97\u5b50<\/strong> \u2014 \u5404\u51fa\u529b\u5217\u306e\u7cfb\u7d71\u306f\u3001\u305d\u308c\u304c\u901a\u904e\u3057\u305f\u5909\u63db\u3092\u8a18\u9332\u3057\u307e\u3059\u3002 <code>\u6bce\u65e5\u306e\u652f\u51fa<\/code> \u300c\u91d1\u984d\u304b\u3089\u300d\u3060\u3051\u3067\u306f\u306a\u304f\u300c <code>\u984d<\/code> \u7d4c\u7531 <code>\u548c<\/code>&#8220;.<\/li><\/ul>\n\n\n\n<p>Under the hood this is the General SQL Parser engine \u2014 a commercial SQL compiler front-end developed since the mid-2000s and validated against roughly 13,600 per-dialect test fixtures. Lineage quality is a parsing problem before it is anything else, and that corpus is what keeps edge-case Spark syntax from silently producing wrong edges.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Spark SQL\u30c7\u30fc\u30bf\u30ea\u30cd\u30fc\u30b8\u304c\u30b9\u30bf\u30c3\u30af\u5185\u3067\u3069\u306e\u3088\u3046\u306b\u4f4d\u7f6e\u3065\u3051\u3089\u308c\u308b\u304b<\/h2>\n\n\n\n<p>Spark\u306f\u5358\u72ec\u3067\u52d5\u4f5c\u3059\u308b\u3053\u3068\u306f\u307b\u3068\u3093\u3069\u3042\u308a\u307e\u305b\u3093\u3002\u305d\u306e\u30c6\u30fc\u30d6\u30eb\u306f\u901a\u5e38Hive\u30e1\u30bf\u30b9\u30c8\u30a2\u306b\u683c\u7d0d\u3055\u308c\u3001Spark\u30ef\u30fc\u30af\u30ed\u30fc\u30c9\u306e\u5927\u90e8\u5206\u306fDatabricks\u4e0a\u3067\u5b9f\u884c\u3055\u308c\u308b\u3088\u3046\u306b\u306a\u3063\u3066\u3044\u307e\u3059\u3002SQLFlow\u306f\u3053\u308c\u3089\u3092\u3001\u305d\u308c\u305e\u308c\u72ec\u7acb\u3057\u3066\u89e3\u6790\u3055\u308c\u308b\u7b2c\u4e00\u7d1a\u306e\u65b9\u8a00\u3068\u3057\u3066\u6271\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<ul><li>Hive\u30b8\u30e7\u30d6\u3092Spark\u306b\u79fb\u884c\u3057\u307e\u3059\u304b\uff1f\u30ec\u30ac\u30b7\u30fcHQL\u3092\u5206\u6790\u3057\u307e\u3059 <a href=\"https:\/\/www.gudusoft.com\/jp\/%e3%83%8f%e3%82%a4%e3%83%96%e3%83%87%e3%83%bc%e3%82%bf%e7%b3%bb%e7%b5%b1\/\">Hive\u30c7\u30fc\u30bf\u30ea\u30cd\u30fc\u30b8\u30d1\u30fc\u30b5\u30fc<\/a> \u305d\u3057\u3066\u3001Spark\u30d1\u30fc\u30b5\u30fc\u3092\u4f7f\u7528\u3057\u3066\u66f8\u304d\u63db\u3048\u3089\u308c\u305f\u30b8\u30e7\u30d6\u3092\u4f5c\u6210\u3057\u30012\u3064\u306e\u4f9d\u5b58\u95a2\u4fc2\u30b0\u30e9\u30d5\u3092\u6bd4\u8f03\u3057\u3066\u3001\u5909\u63db\u6642\u306b\u4f55\u3082\u5931\u308f\u308c\u3066\u3044\u306a\u3044\u3053\u3068\u3092\u78ba\u8a8d\u3057\u307e\u3059\u3002<\/li>\n<li>Databricks\u4e0a\u3067\u5b9f\u884c\u4e2d\u3067\u3059\u304b\uff1f <a href=\"https:\/\/www.gudusoft.com\/jp\/%e3%83%87%e3%83%bc%e3%82%bf%e3%83%96%e3%83%aa%e3%83%83%e3%82%af%e3%82%b9%e3%81%ae%e3%83%87%e3%83%bc%e3%82%bf%e3%83%aa%e3%83%8d%e3%83%bc%e3%82%b8\/\">Databricks \u30c7\u30fc\u30bf\u30ea\u30cd\u30fc\u30b8 \u30d1\u30fc\u30b5\u30fc<\/a> \u672c\u66f8\u306f\u3001\u30aa\u30fc\u30d7\u30f3\u30bd\u30fc\u30b9\u306eSpark SQL\u3068\u306f\u4e56\u96e2\u3057\u3066\u304a\u308a\u3001\u5358\u4e00\u306e\u5171\u901a\u6587\u6cd5\u3067\u306f\u9069\u5207\u306b\u51e6\u7406\u3067\u304d\u306a\u3044Databricks SQL\u65b9\u8a00\u3092\u5bfe\u8c61\u3068\u3057\u3066\u3044\u307e\u3059\u3002<\/li>\n<li>\u6df7\u5408\u74b0\u5883\uff1f SQLFlow \u306f\u30b5\u30dd\u30fc\u30c8\u3055\u308c\u3066\u3044\u308b 39 \u306e\u65b9\u8a00\u3059\u3079\u3066\u3092 1 \u3064\u306e\u7cfb\u7d71\u30ea\u30dd\u30b8\u30c8\u30ea\u306b\u5206\u6790\u3059\u308b\u305f\u3081\u3001Snowflake \u30bf\u30b9\u30af\u306b\u3088\u3063\u3066\u751f\u6210\u3055\u308c\u305f\u30c6\u30fc\u30d6\u30eb\u3092\u8aad\u307f\u53d6\u308b Spark \u30b8\u30e7\u30d6\u306f\u30011 \u3064\u306e\u9023\u7d9a\u3057\u305f\u30b0\u30e9\u30d5\u3068\u3057\u3066\u8868\u793a\u3055\u308c\u307e\u3059\u3002 <a href=\"https:\/\/www.gudusoft.com\/jp\/sql-data-lineage-tool\/\">SQL\u30c7\u30fc\u30bf\u30ea\u30cd\u30fc\u30b8\u30c4\u30fc\u30eb\u306e\u6982\u8981<\/a> \u5168\u4f53\u50cf\u3092\u628a\u63e1\u3059\u308b\u306b\u306f\u3002<\/li><\/ul>\n\n\n\n<p>\u5165\u529b\u65b9\u6cd5\u306f\u67d4\u8edf\u3067\u3001SQL\u306e\u8cbc\u308a\u4ed8\u3051\u3001\u30d5\u30a1\u30a4\u30eb\u306e\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3001JDBC\u7d4c\u7531\u3067\u306e\u30e1\u30bf\u30c7\u30fc\u30bf\u306e\u53d6\u5f97\u3001\u307e\u305f\u306fdbt-on-Spark\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u7528\u306edbt\u30de\u30cb\u30d5\u30a7\u30b9\u30c8\u306e\u30a4\u30f3\u30dd\u30fc\u30c8\u304c\u53ef\u80fd\u3067\u3059\u3002\u51fa\u529b\u306f\u3001\u30a4\u30f3\u30bf\u30e9\u30af\u30c6\u30a3\u30d6\u306a\u30c9\u30ea\u30eb\u30c0\u30a6\u30f3\u53ef\u80fd\u306a\u56f3\u3001JSON\u3001CSV\u3001PNG\u5f62\u5f0f\u306e\u30a8\u30af\u30b9\u30dd\u30fc\u30c8\u3001\u304a\u3088\u3073\u81ea\u52d5\u5316\u306e\u305f\u3081\u306eREST API\u3067\u3059\u3002\u4f8b\u3048\u3070\u3001\u30d7\u30eb\u30ea\u30af\u30a8\u30b9\u30c8\u3067\u5909\u66f4\u3055\u308c\u305fSQL\u3092CI\u30b9\u30c6\u30c3\u30d7\u3068\u3057\u3066\u5206\u6790\u3059\u308b\u3068\u3044\u3063\u305f\u3053\u3068\u304c\u53ef\u80fd\u3067\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u5c55\u958b\u3068\u62e1\u5f35<\/h2>\n\n\n\n<p>SQLFlow Cloud\u306b\u306f\u30a4\u30f3\u30bf\u30e9\u30af\u30c6\u30a3\u30d6\u306b\u4f7f\u7528\u3067\u304d\u308b\u7121\u6599\u30d7\u30e9\u30f3\u304c\u3042\u308a\u3001\u30d7\u30ec\u30df\u30a2\u30e0\u30d7\u30e9\u30f3\u306f\u6708\u984d$49.99\u3067\u3059\u3002SQL\u304c\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u5916\u306b\u51fa\u3089\u308c\u306a\u3044\u30c1\u30fc\u30e0\u306f\u3001 <a href=\"https:\/\/www.gudusoft.com\/jp\/sqlflow-%e3%82%aa%e3%83%b3%e3%83%97%e3%83%ac%e3%83%9f%e3%82%b9-%e3%83%90%e3%83%bc%e3%82%b8%e3%83%a7%e3%83%b3\/\">SQLFlow \u30aa\u30f3\u30d7\u30ec\u30df\u30b9<\/a> \u2014 Docker\u307e\u305f\u306fKubernetes\u306b\u5bfe\u5fdc\u3001\u30a8\u30a2\u30ae\u30e3\u30c3\u30d7\u74b0\u5883\u306b\u3082\u5bfe\u5fdc\u3001\u9078\u629e\u3057\u305f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u30bf\u30a4\u30d7\u3054\u3068\u306b\u6708\u984d$500\u307e\u305f\u306f1\u56de\u9650\u308a\u306e$4,800\u3002\u3044\u305a\u308c\u306e\u5834\u5408\u3082\u3001SQLFlow\u306fSQL\u30b3\u30fc\u30c9\u306e\u9759\u7684\u89e3\u6790\u306e\u307f\u3092\u5b9f\u884c\u3057\u3001\u30c6\u30fc\u30d6\u30eb\u306e\u884c\u3092\u8aad\u307f\u53d6\u308b\u3053\u3068\u306f\u3042\u308a\u307e\u305b\u3093\u3002\u30a8\u30f3\u30bf\u30fc\u30d7\u30e9\u30a4\u30ba\u5c55\u958b\u3067\u306f\u3001\u5897\u5206\u30b9\u30ad\u30e3\u30f3\u3068\u6c38\u7d9a\u7684\u306a\u30ea\u30cd\u30fc\u30b8\u30ea\u30dd\u30b8\u30c8\u30ea\u3092\u4f7f\u7528\u3057\u3066\u3001100\u4ee5\u4e0a\u306e\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u3068100\u4e07\u4ee5\u4e0a\u306e\u5217\u304b\u3089\u306a\u308b\u74b0\u5883\u3092\u30d0\u30c3\u30c1\u30b9\u30ad\u30e3\u30f3\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u3088\u304f\u3042\u308b\u8cea\u554f<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">\u7cfb\u7d71\u60c5\u5831\u3092\u53d6\u5f97\u3059\u308b\u306b\u306f\u3001\u7a3c\u50cd\u4e2d\u306eSpark\u30af\u30e9\u30b9\u30bf\u30fc\u304c\u5fc5\u8981\u3067\u3059\u304b\uff1f<\/h3>\n\n\n<p>\u3044\u3044\u3048\u3002SQLFlow\u306fSpark SQL\u30c6\u30ad\u30b9\u30c8\u3092\u9759\u7684\u306b\u89e3\u6790\u3057\u307e\u3059\u3002\u30d7\u30eb\u30ea\u30af\u30a8\u30b9\u30c8\u3001\u30d9\u30f3\u30c0\u30fc\u304b\u3089\u306e\u7d0d\u54c1\u7269\u3001\u79fb\u884c\u30d0\u30c3\u30af\u30ed\u30b0\u306a\u3069\u3001\u3053\u308c\u307e\u3067\u4e00\u5ea6\u3082\u5b9f\u884c\u3055\u308c\u305f\u3053\u3068\u306e\u306a\u3044\u30b9\u30af\u30ea\u30d7\u30c8\u3092\u89e3\u6790\u3057\u3066\u3082\u3001\u672c\u756a\u30b3\u30fc\u30c9\u304b\u3089\u5f97\u3089\u308c\u308b\u306e\u3068\u540c\u3058\u5217\u30ec\u30d9\u30eb\u306e\u30c7\u30fc\u30bf\u7cfb\u5217\u3092\u53d6\u5f97\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u3053\u308c\u306fOpenLineage\u306eSpark\u7d71\u5408\u3068\u3069\u3046\u9055\u3046\u306e\u3067\u3059\u304b\uff1f<\/h3>\n\n\n<p>OpenLineage\u306f\u3001Spark\u30bb\u30c3\u30b7\u30e7\u30f3\u4e0a\u306e\u30ea\u30b9\u30ca\u30fc\u3092\u4ecb\u3057\u3066\u5b9f\u884c\u6642\u306b\u30ea\u30cd\u30fc\u30b8\u3092\u30ad\u30e3\u30d7\u30c1\u30e3\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u5b9f\u884c\u4e2d\u306e\u30b8\u30e7\u30d6\u304c\u5b9f\u969b\u306b\u4f55\u3092\u3057\u305f\u304b\u3092\u76e3\u8996\u3059\u308b\u306e\u306b\u6700\u9069\u3067\u3059\u3002SQLFlow\u306fSQL\u30b3\u30fc\u30c9\u81ea\u4f53\u304b\u3089\u30ea\u30cd\u30fc\u30b8\u3092\u751f\u6210\u3059\u308b\u305f\u3081\u3001\u5b9f\u884c\u3055\u308c\u3066\u3044\u306a\u3044\u30b3\u30fc\u30c9\u3082\u5bfe\u8c61\u3068\u3057\u3001\u30de\u30fc\u30b8\u524d\u306e\u30ec\u30d3\u30e5\u30fc\u3084\u79fb\u884c\u76e3\u67fb\u3092\u30b5\u30dd\u30fc\u30c8\u3057\u3001\u30af\u30e9\u30b9\u30bf\u3078\u306e\u30a2\u30af\u30bb\u30b9\u3082\u4e0d\u8981\u3067\u3059\u3002\u3053\u306e2\u3064\u306f\u76f8\u4e92\u88dc\u5b8c\u7684\u306a\u95a2\u4fc2\u306b\u3042\u308a\u3001SQLFlow\u306f\u5b9f\u884c\u6642\u306e\u30ea\u30cd\u30fc\u30b8\u3068\u4e26\u884c\u3057\u3066\u3001DataHub\u3001Purview\u3001\u307e\u305f\u306fOpenMetadata\u306b\u30a8\u30af\u30b9\u30dd\u30fc\u30c8\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">SQLFlow\u306f\u4e00\u6642\u30d3\u30e5\u30fc\u3092\u901a\u3057\u3066\u30c7\u30fc\u30bf\u7cfb\u5217\u3092\u8ffd\u8de1\u3067\u304d\u307e\u3059\u304b\uff1f<\/h3>\n\n\n<p>\u306f\u3044\u3002 <code>\u4e00\u6642\u30d3\u30e5\u30fc\u306e\u4f5c\u6210\u307e\u305f\u306f\u7f6e\u63db<\/code> \u30b9\u30c6\u30fc\u30c8\u30e1\u30f3\u30c8\u306f\u30a8\u30f3\u30c9\u30c4\u30fc\u30a8\u30f3\u30c9\u3067\u89e3\u6c7a\u3055\u308c\u308b\u305f\u3081\u3001\u6700\u7d42\u7684\u306aCTAS\u306e\u5217\u306f\u3001\u5404\u30b9\u30c6\u30c3\u30d7\u3067\u95a2\u6570\u3068\u96c6\u8a08\u304c\u9069\u7528\u3055\u308c\u305f\u72b6\u614b\u3067\u3001\u3059\u3079\u3066\u306e\u30b9\u30c6\u30fc\u30b8\u30f3\u30b0\u30d3\u30e5\u30fc\u3092\u305f\u3069\u3063\u3066\u7269\u7406\u7684\u306a\u30bd\u30fc\u30b9\u30c6\u30fc\u30d6\u30eb\u307e\u3067\u30c8\u30ec\u30fc\u30b9\u3055\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Databricks SQL\u306f\u3001\u3053\u3053\u3067\u3044\u3046Spark SQL\u3068\u540c\u3058\u3082\u306e\u3067\u3059\u304b\uff1f<\/h3>\n\n\n<p>\u3044\u3044\u3048\u3002SQLFlow\u306f\u3001\u305d\u308c\u305e\u308c\u7570\u306a\u308b\u65b9\u8a00\u30d1\u30fc\u30b5\u30fc\u3092\u4f7f\u7528\u3057\u3066\u89e3\u6790\u3057\u307e\u3059\u3002Databricks SQL\u306b\u306f\u72ec\u81ea\u306e\u69cb\u6587\u62e1\u5f35\u304c\u3042\u308b\u305f\u3081\u3001\u72ec\u81ea\u306e\u6587\u6cd5\u3092\u6301\u3061\u307e\u3059\u3002Databricks\u4e0a\u3067\u5b9f\u884c\u3059\u308b\u5834\u5408\u306fDatabricks\u65b9\u8a00\u3092\u3001\u30aa\u30fc\u30d7\u30f3\u30bd\u30fc\u30b9\u306eSpark\u3001EMR\u3001\u307e\u305f\u306f\u81ea\u5df1\u7ba1\u7406\u578b\u30af\u30e9\u30b9\u30bf\u30fc\u306e\u5834\u5408\u306fSpark SQL\u65b9\u8a00\u3092\u4f7f\u7528\u3057\u3066\u304f\u3060\u3055\u3044\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">SQLFlow\u306f\u79c1\u306e\u30c7\u30fc\u30bf\u3092\u8aad\u307f\u53d6\u308a\u307e\u3059\u304b\uff1f<\/h3>\n\n\n<p>\u3044\u3044\u3048\u3002SQL\u30b3\u30fc\u30c9\u3068\u3001\u5fc5\u8981\u306b\u5fdc\u3058\u3066\u30b9\u30ad\u30fc\u30de\u30e1\u30bf\u30c7\u30fc\u30bf\uff08\u30c6\u30fc\u30d6\u30eb\u3068\u5217\u306e\u5b9a\u7fa9\uff09\u3092\u5206\u6790\u3057\u307e\u3059\u3002\u884c\u30c7\u30fc\u30bf\u306b\u306f\u4e00\u5207\u30a2\u30af\u30bb\u30b9\u3057\u307e\u305b\u3093\u3002\u30aa\u30f3\u30d7\u30ec\u30df\u30b9\u7248\u3067\u306f\u3001SQL\u30c6\u30ad\u30b9\u30c8\u3082\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u5185\u306b\u7559\u307e\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Spark SQL\u306e\u7cfb\u7d71\u8ffd\u8de1\u3068SQLFlow\u306e\u8cbb\u7528\u306f\u3044\u304f\u3089\u3067\u3059\u304b\uff1f<\/h3>\n\n\n<p>SQLFlow Cloud\u306f\u7121\u6599\u304b\u3089\u30b9\u30bf\u30fc\u30c8\u3002\u30d7\u30ec\u30df\u30a2\u30e0\u30d7\u30e9\u30f3\u306f\u6708\u984d$49.99\u3002\u30aa\u30f3\u30d7\u30ec\u30df\u30b9\u7248\u306f\u6708\u984d$500\u3001\u307e\u305f\u306f\u9078\u629e\u3057\u305f\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u30bf\u30a4\u30d7\u3054\u3068\u306b1\u56de\u9650\u308a\u306e$4,800\u3067\u30012\u53f0\u306e\u30b5\u30fc\u30d0\u30fc\u306b\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u53ef\u80fd\u3002\u8a73\u7d30\u306f\u4ee5\u4e0b\u3092\u53c2\u7167\u3002 <a href=\"https:\/\/www.gudusoft.com\/jp\/%e4%be%a1%e6%a0%bc%e8%a8%ad%e5%ae%9a\/\">\u4fa1\u683c\u8a2d\u5b9a<\/a> \u8a73\u7d30\u306f<\/p>\n\n\n\n<div class=\"wp-block-group alignfull has-background is-layout-constrained\" style=\"background-color:#60d5f6;padding-top:32px;padding-bottom:32px\"><div class=\"wp-block-group__inner-container\">\n\n<h2 class=\"wp-block-heading\">Spark SQL \u3092\u4eca\u3059\u3050\u30c8\u30ec\u30fc\u30b9\u3057\u307e\u3057\u3087\u3046<\/h2>\n\n\n<p>\u7121\u6599\u306e\u30d3\u30b8\u30e5\u30a2\u30e9\u30a4\u30b6\u30fc\u306bSpark SQL\u30b9\u30af\u30ea\u30d7\u30c8\u3092\u8cbc\u308a\u4ed8\u3051\u3066\u3001\u30bd\u30fc\u30b9\u304b\u3089\u30bf\u30fc\u30b2\u30c3\u30c8\u307e\u3067\u306e\u4efb\u610f\u306e\u5217\u3092\u8ffd\u8de1\u3059\u308b\u304b\u3001\u307e\u305f\u306f\u304a\u5ba2\u69d8\u306e\u30b7\u30b9\u30c6\u30e0\u5168\u4f53\u306e\u30b9\u30ad\u30e3\u30f3\u306b\u3064\u3044\u3066\u3054\u76f8\u8ac7\u304f\u3060\u3055\u3044\u3002<\/p>\n\n\n<div class=\"wp-block-buttons is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/sqlflow.gudusoft.com\/?utm_source=gudusoft&amp;utm_medium=website&amp;utm_campaign=spark-sql-data-lineage\" target=\"_blank\" rel=\"noreferrer noopener\">SQLFlow\u3092\u7121\u6599\u3067\u304a\u8a66\u3057\u304f\u3060\u3055\u3044<\/a><\/div>\n<\/div>\n\n\n<div class=\"wp-block-buttons is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.gudusoft.com\/jp\/%e3%82%b3%e3%83%b3%e3%82%bf%e3%82%af%e3%83%88\/\">\u30a8\u30f3\u30bf\u30fc\u30d7\u30e9\u30a4\u30ba\u5411\u3051\u30c7\u30e2\u3092\u30ea\u30af\u30a8\u30b9\u30c8\u3059\u308b<\/a><\/div>\n<\/div>\n\n<\/div><\/div>\n\n\n\n<script type=\"application\/ld+json\">{\n    \"@context\": \"https:\\\/\\\/schema.org\",\n    \"@graph\": [\n        {\n            \"@type\": \"SoftwareApplication\",\n            \"name\": \"Gudu SQLFlow\",\n            \"applicationCategory\": \"DeveloperApplication\",\n            \"applicationSubCategory\": \"SQL Data Lineage Tool\",\n            \"operatingSystem\": \"Web, Linux, Windows, macOS\",\n            \"url\": \"https:\\\/\\\/www.gudusoft.com\\\/spark-sql-data-lineage\\\/\",\n            \"description\": \"Automated Spark SQL data lineage from static parsing: column-level lineage through CTAS, INSERT INTO\\\/OVERWRITE, and temp view chains, with no cluster or job run required.\",\n            \"featureList\": \"Spark SQL dialect parser, column-level lineage, temp view resolution, CTAS and INSERT OVERWRITE analysis, indirect\\\/impact lineage, dbt support, REST API, DataHub\\\/Purview\\\/OpenMetadata export\",\n            \"softwareVersion\": \"8.2.3\",\n            \"offers\": [\n                {\n                    \"@type\": \"Offer\",\n                    \"name\": \"SQLFlow Cloud Free\",\n                    \"price\": \"0\",\n                    \"priceCurrency\": \"USD\"\n                },\n                {\n                    \"@type\": \"Offer\",\n                    \"name\": \"SQLFlow Cloud Premium\",\n                    \"price\": \"49.99\",\n                    \"priceCurrency\": \"USD\",\n                    \"priceSpecification\": {\n                        \"@type\": \"UnitPriceSpecification\",\n                        \"price\": \"49.99\",\n                        \"priceCurrency\": \"USD\",\n                        \"billingIncrement\": 1,\n                        \"unitText\": \"MONTH\"\n                    }\n                },\n                {\n                    \"@type\": \"Offer\",\n                    \"name\": \"SQLFlow On-Premise\",\n                    \"price\": \"4800\",\n                    \"priceCurrency\": \"USD\"\n                }\n            ],\n            \"publisher\": {\n                \"@type\": \"Organization\",\n                \"name\": \"Gudu Software\",\n                \"url\": \"https:\\\/\\\/www.gudusoft.com\\\/\"\n            }\n        },\n        {\n            \"@type\": \"FAQPage\",\n            \"mainEntity\": [\n                {\n                    \"@type\": \"Question\",\n                    \"name\": \"Do I need a running Spark cluster to get lineage?\",\n                    \"acceptedAnswer\": {\n                        \"@type\": \"Answer\",\n                        \"text\": \"No. SQLFlow parses the Spark SQL text statically. You can analyze a script that has never executed anywhere and get the same column-level lineage you would get from production code.\"\n                    }\n                },\n                {\n                    \"@type\": \"Question\",\n                    \"name\": \"How is this different from OpenLineage's Spark integration?\",\n                    \"acceptedAnswer\": {\n                        \"@type\": \"Answer\",\n                        \"text\": \"OpenLineage captures lineage at run time via a listener on the Spark session, which is excellent for observing what live jobs actually did. SQLFlow derives lineage from the SQL code itself, so it also covers code that hasn't run, supports pre-merge review and migration audits, and needs no cluster access. The two are complementary.\"\n                    }\n                },\n                {\n                    \"@type\": \"Question\",\n                    \"name\": \"Can SQLFlow trace lineage through temp views?\",\n                    \"acceptedAnswer\": {\n                        \"@type\": \"Answer\",\n                        \"text\": \"Yes. Chains of CREATE OR REPLACE TEMP VIEW statements are resolved end to end, so a column in a final CTAS traces back through every staging view to the physical source table, with the functions and aggregations applied at each step.\"\n                    }\n                },\n                {\n                    \"@type\": \"Question\",\n                    \"name\": \"Is Databricks SQL the same as Spark SQL here?\",\n                    \"acceptedAnswer\": {\n                        \"@type\": \"Answer\",\n                        \"text\": \"No. SQLFlow parses them with separate dialect parsers. Databricks SQL has its own syntax extensions, so it gets its own grammar. Use the Databricks dialect on Databricks and the Spark SQL dialect for open-source Spark, EMR, or self-managed clusters.\"\n                    }\n                },\n                {\n                    \"@type\": \"Question\",\n                    \"name\": \"Does SQLFlow read my data?\",\n                    \"acceptedAnswer\": {\n                        \"@type\": \"Answer\",\n                        \"text\": \"No. It analyzes SQL code and, optionally, schema metadata such as table and column definitions. Row data is never accessed. With the On-Premise edition, even the SQL text stays inside your network.\"\n                    }\n                },\n                {\n                    \"@type\": \"Question\",\n                    \"name\": \"What does Spark SQL lineage with SQLFlow cost?\",\n                    \"acceptedAnswer\": {\n                        \"@type\": \"Answer\",\n                        \"text\": \"SQLFlow Cloud starts free; premium is $49.99\\\/month. SQLFlow On-Premise is $500\\\/month or $4,800 one-time per selected database type, installable on two servers.\"\n                    }\n                }\n            ]\n        }\n    ]\n}<\/script>","protected":false},"excerpt":{"rendered":"<p>Spark SQL data lineage is the column-level map of how data moves through your Spark SQL code: which source columns feed every table written by a CREATE TABLE &#8230; AS SELECT or INSERT OVERWRITE, and through which temp views, joins, functions, and filters along the way. Gudu SQLFlow builds that map by statically parsing your Spark SQL with a dedicated Spark dialect parser \u2014 no cluster to instrument, no listener to attach, and no job run required. Paste the SQL, get the lineage diagram. Try it now: paste any Spark SQL script into the free online SQL lineage visualizer, select\u2026<\/p>","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"blocksy_meta":{"styles_descriptor":{"styles":{"desktop":"","tablet":"","mobile":""},"google_fonts":[],"version":5}},"_links":{"self":[{"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/pages\/6761"}],"collection":[{"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/comments?post=6761"}],"version-history":[{"count":0,"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/pages\/6761\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.gudusoft.com\/jp\/wp-json\/wp\/v2\/media?parent=6761"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}