{"id":4353,"date":"2024-01-24T09:45:34","date_gmt":"2024-01-24T00:45:34","guid":{"rendered":"https:\/\/blog.since2020.jp\/?p=4353"},"modified":"2024-01-24T10:02:49","modified_gmt":"2024-01-24T01:02:49","slug":"dbt-core%e7%92%b0%e5%a2%83%e6%a7%8b%e7%af%89-for-bigquery","status":"publish","type":"post","link":"https:\/\/since2020.jp\/media\/dbt-core%e7%92%b0%e5%a2%83%e6%a7%8b%e7%af%89-for-bigquery\/","title":{"rendered":"dbt core\u74b0\u5883\u69cb\u7bc9 for BigQuery"},"content":{"rendered":"\n

dbt (data build tool) \u306f\u30c7\u30fc\u30bf\u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u3084\u30c7\u30fc\u30bf\u30a8\u30f3\u30b8\u30cb\u30a2\u304cSQL\u3092\u7528\u3044\u3066\u30c7\u30fc\u30bf\u5909\u63db\u51e6\u7406\uff08Transformation\uff09\u3092\u3088\u308a\u52b9\u7387\u7684\u306b\u884c\u3046\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u3053\u306e\u8a18\u4e8b\u3067\u306f\u3001Google BigQuery\u3092\u5bfe\u8c61\u3068\u3057\u305fdbt core\u306e\u74b0\u5883\u69cb\u7bc9\u30d7\u30ed\u30bb\u30b9\u306b\u3064\u3044\u3066\u8a73\u3057\u304f\u8aac\u660e\u3057\u307e\u3059\u3002<\/p>\n\n\n

\u306f\u3058\u3081\u306b<\/h2>\n

dbt (data build tool) \u306f\u30c7\u30fc\u30bf\u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u3084\u30c7\u30fc\u30bf\u30a8\u30f3\u30b8\u30cb\u30a2\u304cSQL\u3092\u7528\u3044\u3066\u30c7\u30fc\u30bf\u5909\u63db\u51e6\u7406\uff08Transformation\uff09\u3092\u3088\u308a\u52b9\u7387\u7684\u306b\u884c\u3046\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u3053\u306e\u8a18\u4e8b\u3067\u306f\u3001Google BigQuery\u3092\u5bfe\u8c61\u3068\u3057\u305fdbt core\u306e\u74b0\u5883\u69cb\u7bc9\u30d7\u30ed\u30bb\u30b9\u306b\u3064\u3044\u3066\u8a73\u3057\u304f\u8aac\u660e\u3057\u307e\u3059\u3002<\/span><\/p>\n\n

\u30b9\u30c6\u30c3\u30d71: GCP\u306eAPI\u3068\u30b5\u30fc\u30d3\u30b9\u304b\u3089dbt\u7528\u306e\u79d8\u5bc6\u9375\u3092\u4f5c\u6210<\/h2>\n

dbt\u3092BigQuery\u3068\u9023\u643a\u3055\u305b\u308b\u305f\u3081\u306b\u306f\u3001Google Cloud Platform (GCP) \u3067API\u3068\u30b5\u30fc\u30d3\u30b9\u306e\u79d8\u5bc6\u9375\u3092\u4f5c\u6210\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002GCP\u306e\u30b3\u30f3\u30bd\u30fc\u30eb\u306b\u30ed\u30b0\u30a4\u30f3\u3057\u3001\u9069\u5207\u306a\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u3092\u9078\u629e\u3057\u305f\u5f8c\u3001API\u3068\u30b5\u30fc\u30d3\u30b9\u306e\u8a8d\u8a3c\u60c5\u5831\u30da\u30fc\u30b8\u306b\u79fb\u52d5\u3057\u3066\u79d8\u5bc6\u9375\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u3053\u306e\u79d8\u5bc6\u9375\u306f\u3001\u5f8c\u307b\u3069dbt\u3068BigQuery\u3092\u63a5\u7d9a\u3059\u308b\u969b\u306b\u4f7f\u7528\u3057\u307e\u3059\u3002<\/span><\/p>\r\n

\u79d8\u5bc6\u9375\u4f5c\u6210\u306e\u8a73\u7d30\u306f\u3053\u3061\u3089<\/span><\/a><\/span><\/span><\/p>\n\n

\u30b9\u30c6\u30c3\u30d72: \u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306e\u4f5c\u6210<\/h2>\n

\u307e\u305a\u3001\u4efb\u610f\u306e\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u540d\u3067\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u3092\u4f5c\u6210\u3057\u3066\u3001\u4f5c\u6210\u3057\u305f\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306b\u79fb\u52d5\u3057\u307e\u3059\u3002\u3053\u306e\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306f\u3001dbt\u306e\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u30d5\u30a1\u30a4\u30eb\u3092\u4fdd\u6301\u3059\u308b\u5834\u6240\u306b\u306a\u308a\u307e\u3059\u3002<\/span><\/p>\r\n

\r\n
mkdir my_dbt_project\r\ncd my_dbt_project<\/code><\/pre>\r\n<\/div>\r\n

\r\n<\/p>\n\n
\u30b9\u30c6\u30c3\u30d73: Python\u4eee\u60f3\u74b0\u5883\u306e\u8a2d\u5b9a<\/h2>\n
Python\u306e\u4eee\u60f3\u74b0\u5883\u3092\u8a2d\u5b9a\u3059\u308b\u3053\u3068\u3067\u3001dbt\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3084\u4ed6\u306e\u4f9d\u5b58\u95a2\u4fc2\u3092\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u56fa\u6709\u306e\u74b0\u5883\u5185\u306b\u9589\u3058\u8fbc\u3081\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/span><\/p>\r\n
\r\n
python -m venv dbt-env \r\nsource dbt-env\/bin\/activate<\/code><\/pre>\r\n<\/div>\r\n

\r\n<\/p>\n\n
\u30b9\u30c6\u30c3\u30d74: dbt-bigquery\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb<\/h2>\n
dbt-bigquery\u306f\u3001dbt\u304cBigQuery\u3068\u9023\u643a\u3059\u308b\u305f\u3081\u306e\u30d7\u30e9\u30b0\u30a4\u30f3\u3067\u3059\u3002\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3067\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u3057\u307e\u3059\u3002<\/span><\/p>\r\n
\r\n
pip install dbt-bigquery<\/code><\/pre>\r\n<\/div>\r\n

\r\n<\/p>\n\n
\u30b9\u30c6\u30c3\u30d75: \u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u306e\u78ba\u8a8d<\/h2>\n
\u4ee5\u4e0b\u306e\u30b3\u30de\u30f3\u30c9\u3067dbt\u306e\u30a4\u30f3\u30b9\u30c8\u30fc\u30eb\u304c\u6b63\u3057\u304f\u884c\u308f\u308c\u305f\u304b\u78ba\u8a8d\u3057\u307e\u3059\u3002<\/span><\/p>\r\n
\r\n
dbt --version<\/code><\/pre>\r\n<\/div>\r\n
\"\"
\r\n<\/p>\n\n
\u30b9\u30c6\u30c3\u30d76: dbt\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u306e\u521d\u671f\u5316\u3068BigQuery\u306e\u63a5\u7d9a<\/h2>\n
dbt\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u3092\u521d\u671f\u5316\u3057\u307e\u3059\u3002\u3053\u308c\u306b\u3088\u308a\u5bfe\u8a71\u5f62\u5f0f\u3067BigQuery\u3068\u306e\u63a5\u7d9a\u3092\u884c\u3048\u307e\u3059\u3002<\/span><\/p>\r\n
\r\n
dbt init dbt_tutorial<\/code><\/pre>\r\n<\/div>\r\n\u8a73\u7d30\uff08\u4eca\u56de\u306e\u4f8b\uff09<\/span><\/strong><\/b>\r\n
\u30fbWhich database would you like to use?\uff1a[1] bigquery<\/span><\/strong><\/p>\r\n
\u30fbDesired authentication method option (enter a number):[2] service_account<\/span><\/strong><\/p>\r\n
\u30fbkeyfile (\/path\/to\/bigquery\/keyfile.json):\u79d8\u5bc6\u9375\u306e\u30d5\u30a1\u30a4\u30eb\u30d1\u30b9<\/span><\/strong><\/p>\r\n
\u30fbproject (GCP project id):\u30d7\u30ed\u30b8\u30a7\u30af\u30c8ID<\/span><\/strong><\/p>\r\n
\u30fbdataset (the name of your dbt dataset):\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u540d<\/span><\/strong><\/p>\r\n
\u30fbthreads (1 or more):1<\/span><\/strong><\/p>\r\n
\u30fbjob_execution_timeout_seconds [300]:300<\/span><\/strong><\/p>\r\n
\u30fbDesired location option (enter a number):[1] US<\/span><\/strong><\/p>\r\n
\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u304c\u6b63\u3057\u304f\u4f5c\u6210\u3055\u308c\u305f\u3053\u3068\u3092\u78ba\u8a8d\u3059\u308b\u305f\u3081\u306b\u3001\u8a2d\u5b9a\u30d5\u30a1\u30a4\u30eb\u306e\u5185\u5bb9\u3092\u78ba\u8a8d\u3057\u307e\u3059\u3002<\/span><\/p>\r\n
\r\n
cat ~\/.dbt\/profiles.yml<\/code><\/pre>\r\n<\/div>\r\n
\"\"<\/p>\r\n

\r\n<\/p>\r\n
<\/p>\n\n
\u30b9\u30c6\u30c3\u30d77: \u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u3078\u306e\u79fb\u52d5\u3068\u30c6\u30b9\u30c8\u30c6\u30fc\u30d6\u30eb\u306e\u4f5c\u6210<\/h2>\n
\u6b21\u306b\u3001\u4f5c\u6210\u3057\u305fdbt\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u30c7\u30a3\u30ec\u30af\u30c8\u30ea\u306b\u79fb\u52d5\u3057\u3001\u30c6\u30b9\u30c8\u30c6\u30fc\u30d6\u30eb\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<\/span><\/p>\r\n
\r\n
cd dbt_tutorial \r\ndbt seed<\/code><\/pre>\r\n<\/div>\r\n
\"\"
\r\n<\/p>\r\n
\u30b3\u30de\u30f3\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3068\u3001\u4ee5\u4e0b\u306e\u3088\u3046\u306bBigQuery\u5185\u3067\u65b0\u3057\u3044\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u304c\u4f5c\u6210\u3055\u308c\u307e\u3059\u3002<\/span><\/p>\r\n
\"\"<\/p>\n\n
\u307e\u3068\u3081<\/h2>\n
\u4ee5\u4e0a\u3067\u3001BigQuery\u7528\u306edbt core\u74b0\u5883\u306e\u69cb\u7bc9\u306f\u5b8c\u4e86\u3067\u3059\u3002\u3053\u306e\u30d7\u30ed\u30bb\u30b9\u3092\u901a\u3058\u3066\u3001dbt\u3068BigQuery\u306e\u9023\u643a\u3092\u8a2d\u5b9a\u3057\u3001\u52b9\u7387\u7684\u306a\u30c7\u30fc\u30bf\u5909\u63db\u3068\u5206\u6790\u306e\u305f\u3081\u306e\u57fa\u76e4\u3092\u69cb\u7bc9\u3057\u307e\u3057\u305f\u3002\u3053\u308c\u306b\u3088\u308a\u3001dbt\u3092\u4f7f\u7528\u3057\u3066\u30c7\u30fc\u30bf\u30e2\u30c7\u30ea\u30f3\u30b0\u3001\u30c6\u30b9\u30c8\u3001\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u5316\u306a\u3069\u3092\u884c\u3044\u3001\u30c7\u30fc\u30bf\u5206\u6790\u306e\u7cbe\u5ea6\u3068\u52b9\u7387\u3092\u9ad8\u3081\u3066\u3044\u304f\u3053\u3068\u304c\u53ef\u80fd\u3067\u3059\u3002Happy Data Modeling!<\/span><\/p>","protected":false},"excerpt":{"rendered":"
dbt (data build tool) \u306f\u30c7\u30fc\u30bf\u30b5\u30a4\u30a8\u30f3\u30c6\u30a3\u30b9\u30c8\u3084\u30c7\u30fc\u30bf\u30a8\u30f3\u30b8\u30cb\u30a2\u304cSQL\u3092\u7528\u3044\u3066\u30c7\u30fc\u30bf\u5909\u63db\u51e6\u7406\uff08Transformation\uff09\u3092\u3088\u308a\u52b9\u7387\u7684\u306b\u884c\u3046\u305f\u3081\u306e\u30c4\u30fc\u30eb\u3067\u3059\u3002\u3053\u306e\u8a18\u4e8b\u3067\u306f\u3001Google BigQ […]<\/p>\n","protected":false},"author":87,"featured_media":4358,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","swell_btn_cv_data":"","footnotes":"","_wp_rev_ctl_limit":""},"categories":[1246],"tags":[108,491,490,486,544,151],"class_list":["post-4353","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-infrastructure","tag-bigquery","tag-dbt","tag-elt","tag-486","tag-544","tag-151"],"_links":{"self":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/4353","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/users\/87"}],"replies":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/comments?post=4353"}],"version-history":[{"count":0,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/4353\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media\/4358"}],"wp:attachment":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media?parent=4353"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/categories?post=4353"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/tags?post=4353"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}