{"id":4548,"date":"2024-02-16T11:31:29","date_gmt":"2024-02-16T02:31:29","guid":{"rendered":"https:\/\/blog.since2020.jp\/?p=4548"},"modified":"2024-02-21T11:25:29","modified_gmt":"2024-02-21T02:25:29","slug":"how_to_rag","status":"publish","type":"post","link":"https:\/\/since2020.jp\/media\/how_to_rag\/","title":{"rendered":"\u3010\u521d\u5fc3\u8005\u3067\u3082\u5b9f\u88c5\u53ef\u80fd\u3011OpenAI API\u3068GCP\u74b0\u5883\u3067\u306eRAG\u306e\u5b9f\u88c5\u65b9\u6cd5\u3092\u5fb9\u5e95\u89e3\u8aac\uff01"},"content":{"rendered":"\n<p>\u672c\u8a18\u4e8b\u3067\u306f\u3001\u5b9f\u969b\u306b\u5916\u90e8\u30c7\u30fc\u30bf\u3092\u7528\u610f\u3057\u3001RAG\u306e\u5b9f\u88c5\u30d5\u30ed\u30fc\u3092\u8a73\u3057\u304f\u8aac\u660e\u3057\u3001GCP\u74b0\u5883\u3092\u4f7f\u7528\u3057\u305f\u958b\u767a\u74b0\u5883\u306e\u69cb\u7bc9\u304b\u3089\u3001\u30b7\u30b9\u30c6\u30e0\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u3001\u30c6\u30ad\u30b9\u30c8\u306e\u62bd\u51fa\u3001\u30a8\u30f3\u30d9\u30c7\u30a3\u30f3\u30b0\u51e6\u7406\u3001\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210\u307e\u3067\u306e\u30d7\u30ed\u30bb\u30b9\u3092\u89e3\u8aac\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n<h2>RAG\u3068\u306f\uff1f\uff1f<\/h2>\n<p>RAG\uff08Retrieval-Augmented Generation\uff09\u3068\u306f\u3001\u81ea\u7136\u8a00\u8a9e\u51e6\u7406\uff08NLP\uff09\u306e\u5206\u91ce\u3067\u4f7f\u308f\u308c\u308b\u6280\u8853\u306e\u4e00\u3064\u3067\u3001\u7279\u306b\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\uff08LLM\uff1aLarge Language Models\uff09\u3092\u5f37\u5316\u3059\u308b\u305f\u3081\u306b\u958b\u767a\u3055\u308c\u307e\u3057\u305f\u3002\u3053\u306e\u6280\u8853\u306f\u3001\u8a00\u8a9e\u30e2\u30c7\u30eb\u304c\u60c5\u5831\u3092\u751f\u6210\u3059\u308b\u969b\u306b\u3001\u5358\u306b\u5185\u90e8\u306e\u77e5\u8b58\u3092\u4f7f\u3046\u3060\u3051\u3067\u306a\u304f\u3001\u5916\u90e8\u304b\u3089\u7279\u5b9a\u306e\u60c5\u5831\u3092\u300c\u691c\u7d22\uff08Retrieval\uff09\u300d\u3057\u3066\u304d\u3066\u3001\u305d\u308c\u3092\u5143\u306b\u300c\u751f\u6210\uff08Generation\uff09\u300d\u3059\u308b\u3053\u3068\u3092\u53ef\u80fd\u306b\u3057\u307e\u3059\u3002RAG\u3092\u4f7f\u7528\u3059\u308b\u3053\u3068\u3067\u4e0b\u8a18\u306e\u3088\u3046\u306a\u30e1\u30ea\u30c3\u30c8\u3092\u53d7\u3051\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<\/p>\r\n<ul>\r\n\t<li><strong>\u6700\u65b0\u60c5\u5831\u306e\u6d3b\u7528<\/strong>: RAG\u306f\u5916\u90e8\u304b\u3089\u60c5\u5831\u3092\u53d6\u5f97\u3059\u308b\u305f\u3081\u3001\u5927\u898f\u6a21\u8a00\u8a9e\u30e2\u30c7\u30eb\u304c\u8a13\u7df4\u3055\u308c\u305f\u6642\u70b9\u306e\u77e5\u8b58\u306b\u9650\u3089\u305a\u3001\u6700\u65b0\u306e\u60c5\u5831\u3092\u53cd\u6620\u3059\u308b\u3053\u3068\u304c\u53ef\u80fd\u3067\u3059\u3002<\/li>\r\n\t<li><strong>\u60c5\u5831\u306e\u6b63\u78ba\u6027\u5411\u4e0a<\/strong>: \u691c\u7d22\u306b\u3088\u3063\u3066\u5f97\u3089\u308c\u305f\u60c5\u5831\u3092\u57fa\u306b\u751f\u6210\u3059\u308b\u305f\u3081\u3001\u7279\u5b9a\u306e\u4e8b\u5b9f\u3084\u30c7\u30fc\u30bf\u306b\u57fa\u3065\u3044\u305f\u3088\u308a\u6b63\u78ba\u306a\u56de\u7b54\u304c\u53ef\u80fd\u306b\u306a\u308a\u307e\u3059\u3002<\/li>\r\n\t<li><strong>\u5fdc\u7528\u7bc4\u56f2\u306e\u62e1\u5927<\/strong>: \u69d8\u3005\u306a\u30c7\u30fc\u30bf\u30d9\u30fc\u30b9\u3084\u77e5\u8b58\u6e90\u306b\u30a2\u30af\u30bb\u30b9\u3059\u308b\u3053\u3068\u3067\u3001\u5e45\u5e83\u3044\u5206\u91ce\u306b\u5bfe\u5fdc\u3067\u304d\u308b\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<\/li>\r\n<\/ul>\r\n<p><!-- notionvc: f446db9b-f7fa-4f16-a2c3-ac61e9e8b4ec --><\/p>\n\n<h2>\u4eca\u56de\u5b9f\u88c5\u3059\u308b\u30b5\u30f3\u30d7\u30eb<\/h2>\n<p>\u4eca\u56de\u306fRAG\u306e\u4e00\u9023\u306e\u5b9f\u88c5\u30d5\u30ed\u30fc\u3092\u89e3\u8aac\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u4eca\u56de\u3001\u5916\u90e8\u30c7\u30fc\u30bf\u306fAWS\u304c\u63d0\u4f9b\u3059\u308b\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306e\u7a3c\u50cd\u72b6\u6cc1\u3084\u5229\u7528\u91d1\u984d\u306a\u3069\u304c\u793a\u3055\u308c\u3066\u3044\u308bCUR\uff08Cost and Usage Reports\uff09\u3068\u547c\u3070\u308c\u308b\u30c7\u30fc\u30bf\u306e\u30ea\u30d5\u30a1\u30ec\u30f3\u30b9\u3092\u5bfe\u8c61\u306b\u4e00\u9023\u306e\u30d5\u30ed\u30fc\u3092\u5b9f\u88c5\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u5bfe\u8c61\u306e\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u306f\u4e0b\u8a18\u30ea\u30f3\u30af\u306e\u3082\u306e\u3092\u63a1\u7528\u3057\u307e\u3057\u305f\u3002<!-- notionvc: b4e4dbd1-7658-4dcb-89c6-bda54f73d042 --><\/p>\r\n<p><!-- notionvc: fa343677-38fe-4609-ab67-cb5efe36aa9a --><\/p>\r\n<p>https:\/\/docs.aws.amazon.com\/pdfs\/cur\/latest\/userguide\/cur-user-guide.pdf<\/p>\r\n<p><!-- notionvc: fa343677-38fe-4609-ab67-cb5efe36aa9a --><\/p>\n\n<h2>\u958b\u767a\u74b0\u5883\u306b\u3064\u3044\u3066<\/h2>\n<p>\u4eca\u56de\u3001GCP\u74b0\u5883\u306e\u4e0b\u8a18\u30b5\u30fc\u30d3\u30b9\u3092\u5229\u7528\u3057\u3066RAG\u306e\u5b9f\u88c5\u3092\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u4ed6\u30af\u30e9\u30a6\u30c9\uff08AWS\u3001Azure\uff09\u3067\u3082\u4ee3\u66ff\u30b5\u30fc\u30d3\u30b9\u306f\u7528\u610f\u3055\u308c\u3066\u3044\u308b\u306e\u3067\u53c2\u8003\u306b\u3057\u3066\u304f\u3060\u3055\u3044\u3002<!-- notionvc: d69c0ee2-93eb-4cca-957e-af8b558c81e3 --><\/p>\r\n<p><img decoding=\"async\" src=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2024\/02\/rag_env.png\" \/><\/p>\n\n<h2>\u5b9f\u88c5\u30d5\u30ed\u30fc\u2460 \u301c\u30b7\u30b9\u30c6\u30e0\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u301c<\/h2>\n<p>\u4eca\u56de\u5b9f\u88c5\u3057\u305f\u30d5\u30ed\u30fc\u306e\u30b7\u30b9\u30c6\u30e0\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u306f\u4e0b\u8a18\u306b\u306a\u308a\u307e\u3059\u3002\u5927\u304d\u304f\uff12\u3064\u306e\u30d5\u30ed\u30fc\u306b\u3088\u308a\u69cb\u6210\u3055\u308c\u3066\u3044\u307e\u3059\u3002\uff11\u3064\u76ee\u306f\u3001\u5916\u90e8\u30c7\u30fc\u30bf\u3092\u30d9\u30af\u30c8\u30eb\u5316\u3057Vector DB\u3068\u3057\u3066\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u3067\u304d\u308b\u3088\u3046\u306b\u6e96\u5099\u3059\u308b\u30d5\u30ed\u30fc\u3067\u3059\u3002\uff12\u3064\u76ee\u306f\u3001\u5165\u529b\u3055\u308c\u305f\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u305d\u308c\u306b\u95a2\u9023\u3057\u305f\u30c6\u30ad\u30b9\u30c8\u3092\u691c\u7d22\u3057\u305d\u308c\u3089\u3092\u30c1\u30a7\u30a4\u30f3\u3057\u3066LLM\u306b\u5165\u529b\u3055\u305b\u308b\u30d5\u30ed\u30fc\u3067\u3059\u3002\u5b9f\u969b\u306b\u30a2\u30d7\u30ea\u30b1\u30fc\u30b7\u30e7\u30f3\u3068\u3057\u3066\u904b\u7528\u3059\u308b\u5834\u5408\u306f\u3001\uff12\u3064\u76ee\u306e\u30d5\u30ed\u30fc\u306e\u307f\u3092\u7a3c\u50cd\u3055\u305b\u3001\uff11\u3064\u76ee\u306e\u30d5\u30ed\u30fc\u306f\u5916\u90e8\u30c7\u30fc\u30bf\u3092\u66f4\u65b0\u3059\u308b\u6642\u306e\u307f\u5b9f\u884c\u3059\u308b\u30a4\u30e1\u30fc\u30b8\u3067\u3059\u3002<!-- notionvc: 2a1c92a6-11b8-478f-8763-68f23f0bbbbb --><\/p>\r\n<p><img decoding=\"async\" src=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2024\/02\/arch.png\" \/><!-- notionvc: 1eb1db77-9382-4e3c-af85-fbdff66bc957 --><\/p>\n\n<h2>\u5b9f\u88c5\u30d5\u30ed\u30fc\u2461 \u301c\u30c6\u30ad\u30b9\u30c8\u306e\u62bd\u51fa\u301c<\/h2>\n<p>\u307e\u305a\u306f\u3001\u30d9\u30af\u30c8\u30eb\u5316\u3057\u305f\u3044\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u306e\u30c6\u30ad\u30b9\u30c8\u3092\u62bd\u51fa\u3057\u3001\u30af\u30ec\u30f3\u30b8\u30f3\u30b0\u51e6\u7406\u304b\u3089\u5b9f\u88c5\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u4e0b\u8a18\u304c\u5b9f\u88c5\u4f8b\u3067\u3059\u3002\u4e0b\u8a18\u30d7\u30ed\u30b0\u30e9\u30e0\u306b\u3088\u308a\u3001cur-user-guide.pdf\u304b\u3089\u30c6\u30ad\u30b9\u30c8\u3092\u62bd\u51fa\u3057\u3001\u4e0d\u8981\u306a\u6587\u5b57\u5217\u3092\u524a\u9664\u3057\u30af\u30ec\u30f3\u30b8\u30f3\u30b0\u3057\u307e\u3059\u3002<br \/>\r\n<!-- notionvc: d6cda96f-6468-4e84-bd8a-9e1cf4f7fef7 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>import pandas as pd \r\nfrom pdfminer.high_level import extract_text \r\nimport uuid import re \r\n\r\n# \u30c7\u30fc\u30bf\u5217\u89e3\u8aac\u30da\u30fc\u30b8\u3092\u62bd\u51fa \r\ntext = extract_text('..\/data\/cur-user-guide.pdf', page_numbers=list(range(8, 261))) \r\n\r\n# \u4e0d\u8981\u306a\u6587\u5b57\u5217\u306e\u524a\u9664 \r\ndef drop_text(text): \r\n\u3000\u3000text = text.replace('\\n\\n', '\\n') \r\n\u3000\u3000text = text.replace('\\n', ' ') \r\n\u3000\u3000text = text.replace('AWS Data Exports', '') \r\n\u3000\u3000text = text.replace('User Guide', '') \r\n\u3000\u3000text = text.replace('A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | VWXYZ', '') \r\n\u3000\u3000text = text.replace('Topic', '') text = text.replace('Note', '') text = text.replace('Importtant', '') \r\n\u3000\u3000text = text.replace('\\x0c', '') text = re.sub(r' {3,4}', ' ', text) text = re.sub(r'(?&lt;=\\w) (?=\\w)', '', text) \r\n\u3000\u3000return text \r\n\r\ntext = drop_text(text) \r\n# \u62bd\u51fa\u3057\u305f\u30c6\u30ad\u30b9\u30c8\u3092\u30d4\u30ea\u30aa\u30c9\u3092\u57fa\u6e96\u306b\uff11\u6587\u305a\u3064\u5207\u308a\u51fa\u3057\u3066\u30ea\u30b9\u30c8\u306b\u683c\u7d0d \r\ncur_guide_text = [part + '.' for part in text.split('.') if part] \r\n\r\n# \u610f\u5473\u306e\u306a\u3044\u30c6\u30ad\u30b9\u30c8\u3092\u524a\u9664\u3059\u308b\u305f\u3081\u306b\u6587\u5b57\u6570\u304c10\u5b57\u4ee5\u4e0a\u306e\u30c6\u30ad\u30b9\u30c8\u4ee5\u5916\u306f\u524a\u9664 \r\ncur_guide_text = [s for s in cur_guide_text if len(s) &gt; 10] \r\n\r\n# \u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u4f5c\u6210\u6642\u306b\u5fc5\u8981\u306aid\u5217\u3092\u4f5c\u6210 \r\ndf = pd.DataFrame() df['ID'] = [str(uuid.uuid4()) for _ in range(len(cur_guide_text))] \r\ndf['cur_guide_text'] = cur_guide_text \r\ndf.to_pickle('..\/data\/cur_guide_eng_df.pkl')\r\n<\/code><\/pre>\r\n<p>\u4e0a\u8a18\u30d7\u30ed\u30b0\u30e9\u30e0\u306e\u30dd\u30a4\u30f3\u30c8\u306f\u3001\u30c7\u30fc\u30bf\u30af\u30ec\u30f3\u30b8\u30f3\u30b0\u306e\u6700\u5f8c\u306bid\u5217\u3092\u4f5c\u6210\u3057\u3066\u3044\u308b\u70b9\u3067\u3059\u3002Vertex AI Vector Search\u3092\u5229\u7528\u3057\u3066\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u4f5c\u6210\u3059\u308b\u5834\u5408\u3001id\u3068\u30d9\u30af\u30c8\u30eb\u914d\u5217\u3067\u5bfe\u5fdc\u3055\u308c\u305fjson\u30c7\u30fc\u30bf\u304c\u5fc5\u8981\u3067\u3059\u3002\u305d\u306e\u305f\u3081\u3001\u30c7\u30fc\u30bf\u30af\u30ec\u30f3\u30b8\u30f3\u30b0\u5f8c\u306b\u91cd\u8907\u304c\u51fa\u306a\u3044\u3088\u3046UUID\u3067id\u3092\u4f5c\u6210\u3057\u3066\u3044\u307e\u3059\u3002<!-- notionvc: 845eeb19-9c1a-42b9-a1fc-0baffe85974f --><\/p>\r\n<\/div>\n\n<h2>\u5b9f\u88c5\u30d5\u30ed\u30fc\u2462 \u301c\u30a8\u30f3\u3079\u30c7\u30a3\u30f3\u30b0\u51e6\u7406\u301c<\/h2>\n<p>\u30c6\u30ad\u30b9\u30c8\u306e\u62bd\u51fa\u3068\u524d\u51e6\u7406\u304c\u7d42\u308f\u3063\u305f\u3089\u3001\u6b21\u306f\u30a8\u30f3\u3079\u30c7\u30a3\u30f3\u30b0\u51e6\u7406\u3092\u5b9f\u88c5\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u307e\u305a\u306f\u4e0b\u8a18\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u30a4\u30f3\u30dd\u30fc\u30c8\u3057\u307e\u3059\u3002<\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>from openai import OpenAI \r\nimport pandas as pd \r\nfrom google.oauth2 import service_account \r\nfrom google.cloud import bigquery \r\nimport numpy as np \r\nfrom tqdm import tqdm \r\nimport json \r\nfrom typing import List \r\nimport os<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u3001\u4e0b\u8a18\u306e\u3088\u3046\u306bAPI\u30ad\u30fc\u306e\u8a2d\u5b9a\u3068\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u30a4\u30f3\u30dd\u30fc\u30c8\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code># API\u30ad\u30fc\u3092\u8a2d\u5b9a \r\napi_key = '&lt; OpenAI\u306eAPI\u30ad\u30fc\u3092\u8a2d\u5b9a &gt;' \r\nos.environ[\"OPENAI_API_KEY\"] = api_key \r\n\r\n# OpenAI\u306e\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u4f5c\u6210 \r\nopenai_client = OpenAI() \r\n\r\n# id\u3068\u524d\u51e6\u7406\u3092\u65bd\u3057\u305f\u30c6\u30ad\u30b9\u30c8\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u30a4\u30f3\u30dd\u30fc\u30c8 \r\ndf = pd.read_pickle('..\/data\/cur_guide_eng_df.pkl')<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u30a8\u30f3\u3079\u30c7\u30a3\u30f3\u30b0\u90e8\u5206\u3092\u5b9f\u88c5\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u3053\u3053\u3067\u3082Vertex AI Vector Search\u306e\u5229\u7528\u306b\u5411\u3051\u305f\u30c7\u30fc\u30bf\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u306b\u52a0\u5de5\u3057\u3066\u5b9f\u88c5\u3057\u307e\u3059\u3002\u6700\u7d42\u7684\u306bid\u3068\u30d9\u30af\u30c8\u30eb\u914d\u5217\u304c\u5bfe\u5fdc\u3057\u305fjson\u3092\u51fa\u529b\u3057\u305f\u3044\u306e\u3067\u3001\u4e0b\u8a18\u306e\u3088\u3046\u306b\u5b9f\u88c5\u3057\u307e\u3057\u305f\u3002<br \/>\r\n<br \/>\r\n<\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>with open('..\/output\/cur_guide_eng_vectors.json', 'w') as f: \r\n\u3000\u3000for loop_num in tqdm(range(df.shape[0])): \r\n\u3000\u3000\u3000\u3000detail_text = df['cur_guide_text'][loop_num] \r\n\u3000\u3000\u3000\u3000id = df['ID'][loop_num] \r\n\r\n\u3000\u3000\u3000\u3000# \u3053\u306e\u90e8\u5206\u3067\u30c6\u30ad\u30b9\u30c8\u3092\u30d9\u30af\u30c8\u30eb\u5316\u3057\u3066\u3044\u308b \r\n\u3000\u3000\u3000\u3000vector = openai_client.embeddings.create(model=\"text-embedding-3-large\", input=detail_text).data[0].embedding \r\n\r\n\u3000\u3000\u3000\u3000json_record = {'id':id, 'embedding': vector} \r\n\u3000\u3000\u3000\u3000f.write(json.dumps(json_record)) \r\n\u3000\u3000\u3000\u3000f.write('\\n')<\/code><\/pre>\r\n<\/div>\r\n<p>OpenAI\u306eEmbeddings API\u3067\u306fsmall\u3068large\u306e\uff12\u3064\u306e\u30e2\u30c7\u30eb\u304c\u7528\u610f\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u4eca\u56de\u306f\u7cbe\u5ea6\u306e\u9ad8\u3044large\u30e2\u30c7\u30eb\u3092\u63a1\u7528\u3057\u3066\u3044\u307e\u3059\u304c\u3001\u5fc5\u8981\u306b\u5fdc\u3058\u3066\u4f7f\u3044\u5206\u3051\u3066\u4e0b\u3055\u3044\u3002<\/p>\r\n<blockquote>\r\n<p><img decoding=\"async\" src=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2024\/02\/openai_embedding.png\" width=\"367\" height=\"155\" class=\"\" \/><\/p>\r\n<p>\u5f15\u7528\u5143\uff1ahttps:\/\/platform.openai.com\/docs\/guides\/embeddings\/use-cases<\/p>\r\n<\/blockquote>\r\n<p>\u5148\u8ff0\u306e\u30b3\u30fc\u30c9\u3067\u3001Vertex AI Vector Search\u3067\u5fc5\u8981\u306ajson\u30d5\u30a1\u30a4\u30eb\u306e\u6e96\u5099\u306f\u3067\u304d\u307e\u3057\u305f\u3002\u3057\u304b\u3057\u3001\u3053\u306ejson\u30d5\u30a1\u30a4\u30eb\u306b\u306fid\u3068\u30d9\u30af\u30c8\u30eb\u5024\u3057\u304b\u3042\u308a\u307e\u305b\u3093\u3002\u305d\u306e\u305f\u3081\u3001\u5165\u529b\u306e\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u985e\u4f3c\u3057\u3066\u3044\u308b\u30d9\u30af\u30c8\u30eb\u304c\u3069\u306e\u30c6\u30ad\u30b9\u30c8\u3068\u5bfe\u5fdc\u3057\u3066\u3044\u308b\u306e\u304b\u63a2\u305b\u308b\u3088\u3046\u306b\u3001id\u3001\u30d9\u30af\u30c8\u30eb\u3001\u30c6\u30ad\u30b9\u30c8\u304c\u5bfe\u5fdc\u3057\u305f\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092BigQuery\u306b\u4fdd\u5b58\u3057\u307e\u3059\u3002\u4e0b\u8a18\u306e\u3088\u3046\u306bBigQuery\u306b\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u4fdd\u5b58\u3057\u307e\u3059\u3002<\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>def save_df_to_bq(df, table_id): \r\n\u3000\u3000service_account_key = '&lt; BigQuery\u306b\u30a2\u30af\u30bb\u30b9\u6a29\u306e\u3042\u308b\u30b5\u30fc\u30d3\u30b9\u30a2\u30ab\u30a6\u30f3\u30c8\u306ecredential\u3092\u5b9a\u7fa9 &gt;' \r\n\u3000\u3000credentials = service_account.Credentials.from_service_account_file(service_account_key) \r\n\u3000\u3000bq_client = bigquery.Client(credentials=credentials) \r\n\u3000\u3000job_config = bigquery.QueryJobConfig(write_disposition='WRITE_APPEND') \r\n\u3000\u3000job = bq_client.load_table_from_dataframe(df, table_id) \r\n\u3000\u3000job.result() \r\n\r\ntable_id = '&lt; \u30c7\u30fc\u30bf\u306e\u4fdd\u5b58\u5148\u306e\u30c6\u30fc\u30d6\u30ebid\u3092\u6307\u5b9a &gt;' \r\nsave_df_to_bq(df, table_id)<\/code><\/pre>\r\n<\/div>\r\n<p>&nbsp;<\/p>\n\n<h2>\u5b9f\u88c5\u30d5\u30ed\u30fc\u2463 \u301c\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210\u301c<\/h2>\n<p>\u7d9a\u3044\u3066\u3001Vertex AI Vector Search\u3092\u4f7f\u3063\u3066\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u4f5c\u6210\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u307e\u305a\u306f\u3001\u4e0b\u8a18\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u30a4\u30f3\u30dd\u30fc\u30c8\u3057\u307e\u3059\u3002<!-- notionvc: ca77207a-9ffe-449a-9cf9-4d33df8b8fa2 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>import pandas as pd \r\nfrom google.oauth2 import service_account \r\nimport json \r\nimport subprocess \r\nfrom google.cloud import aiplatform \r\nfrom google.cloud import storage<\/code><\/pre>\r\n<\/div>\r\n<p>\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210\u306f\u3001GCP\u306eVertex AI Vector Search\u3068Cloud Storage\u3092\u5229\u7528\u3057\u307e\u3059\u3002\u305d\u306e\u305f\u3081\u3001\u307e\u305aGCP\u74b0\u5883\u306e\u521d\u671f\u8a2d\u5b9a\u3092\u884c\u3044\u307e\u3059\u3002\u5177\u4f53\u7684\u306b\u306f\u30b5\u30fc\u30d3\u30b9\u30a2\u30ab\u30a6\u30f3\u30c8\u306e\u30a2\u30af\u30c6\u30a3\u30d9\u30a4\u30c8\u3068GCP\u74b0\u5883\u306eproject_id\u3068location\u306e\u53d6\u5f97\u3092\u884c\u3044\u307e\u3059\u3002\u4e0b\u8a18\u3092\u5b9f\u884c\u3059\u308b\u3053\u3068\u3067project_id, location, credentials\u304c\u53d6\u5f97\u3067\u304d\u307e\u3059\u3002<!-- notionvc: 4d1278a9-198e-48be-9540-633b7c17a2a5 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>def load_credentials(): \r\n\u3000\u3000key_file_path = '&lt; \u30b5\u30fc\u30d3\u30b9\u30a2\u30ab\u30a6\u30f3\u30c8\u306ecredential\u3092\u5b9a\u7fa9 &gt;' \r\n\u3000\u3000credentials = service_account.Credentials.from_service_account_file(key_file_path) \r\n\u3000\u3000return key_file_path, credentials \r\n\r\ndef init_gcp(): \r\n\u3000\u3000key_file_path, credentials = load_credentials() \r\n\r\n\u3000\u3000# \u30b5\u30fc\u30d3\u30b9\u30a2\u30ab\u30a6\u30f3\u30c8\u306e\u30a2\u30af\u30c6\u30a3\u30d9\u30a4\u30c8 \r\n\u3000\u3000cmd = ['gcloud', 'auth', 'activate-service-account', '--key-file={}'.format(key_file_path)] \r\n\u3000\u3000result = subprocess.run(cmd, check=True, capture_output=True, text=True) \r\n\r\n\u3000\u3000# PROJECT ID\u306e\u53d6\u5f97 cmd = ['gcloud', 'config', 'get-value', 'project'] \r\n\u3000\u3000PROJECT_ID = subprocess.run(cmd, check=True, capture_output=True, text=True).stdout.strip() \r\n\u3000\u3000LOCATION = \"asia-northeast1\" \r\n\u3000\u3000return PROJECT_ID, LOCATION, credentials \r\n\r\nproject_id, location, credentials = init_gcp()<\/code><\/pre>\r\n<\/div>\r\n<p>\u7d9a\u3044\u3066\u3001\u4eca\u56de\u5229\u7528\u3059\u308b\u30c4\u30fc\u30eb\u306eAPI\u3092\u6709\u52b9\u5316\u3057\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3053\u3068\u3067Vertex AI\u3068Cloud Storage\u3092API\u3067\u547c\u3073\u51fa\u3059\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<!-- notionvc: c4773c21-3ca9-4ecb-957c-51c37aa17c4b --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>cmd = ['gcloud', 'services', 'enable', 'compute.googleapis.com', 'aiplatform.googleapis.com', 'storage.googleapis.com', '--project', project_id] \r\nresult = subprocess.run(cmd, check=True, capture_output=True, text=True)<\/code><\/pre>\r\n<\/div>\r\n<p>\u7d9a\u3044\u3066\u3001\u5148\u307b\u3069\u4f5c\u6210\u3057\u305fid\u3068\u30c6\u30ad\u30b9\u30c8\u306e\u30d9\u30af\u30c8\u30eb\u304c\u5bfe\u5fdc\u3057\u305fjson\u30d5\u30a1\u30a4\u30eb\u3092Cloud Storage\u306e\u6307\u5b9a\u306e\u30d0\u30b1\u30c3\u30c8\u306b\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3053\u3068\u3067\u5148\u307b\u3069\u4f5c\u6210\u3057\u305fjson\u30d5\u30a1\u30a4\u30eb\u3092\u6307\u5b9a\u306e\u30d0\u30b1\u30c3\u30c8\u5185\u306b\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<!-- notionvc: b7035a65-5247-48a9-a3ab-bee1ee061a21 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>def upload_blob(bucket_name, source_file_name, destination_blob_name, credentials): \r\n\u3000\u3000storage_client = storage.Client(credentials= credentials) \r\n\u3000\u3000bucket = storage_client.bucket(bucket_name) \r\n\u3000\u3000blob = bucket.blob(destination_blob_name) \r\n\u3000\u3000if blob.exists(): \r\n\u3000\u3000\u3000blob.delete() \r\n\u3000\u3000generation_match_precondition = 0 \r\n\u3000\u3000blob.upload_from_filename(source_file_name, if_generation_match=generation_match_precondition) \r\n\u3000\u3000print(f\"File {source_file_name} uploaded to {destination_blob_name}.\") \r\n\r\nbucket_name = '&lt; \u30d0\u30b1\u30c3\u30c8\u540d &gt;' \r\nsource_file_name = ' &lt;\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3057\u305f\u3044\u30d5\u30a1\u30a4\u30eb\u306e\u30d1\u30b9&gt; ' \r\ndestination_blob_name = '&lt; \u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u5148\u306e\u30d1\u30b9 &gt;' \r\nupload_blob(bucket_name, source_file_name, destination_blob_name, credentials)<\/code><\/pre>\r\n<\/div>\r\n<p>\u7d9a\u3044\u3066\u3001Vertex AI Vector Search\u3092\u4f7f\u3063\u3066\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u4f5c\u6210\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u307e\u305a\u306f\u4eca\u56de\u5229\u7528\u3059\u308bVertex AI\u306eAI platform\u306e\u521d\u671f\u5316\u3092\u884c\u3044\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3057\u3066\u521d\u671f\u5316\u3092\u884c\u306a\u3063\u3066\u4e0b\u3055\u3044\u3002<!-- notionvc: ac8c53b2-e73a-473e-844d-a64ee52bda95 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code># AI platform\u306e\u521d\u671f\u5316 \r\naiplatform.init(project=project_id, location=location, credentials=credentials)<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u3001\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3068\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u4f5c\u6210\u3067\u304d\u307e\u3059\uff08\u5c11\u3057\u6642\u9593\u304c\u304b\u304b\u308a\u307e\u3059\uff09\u3002aiplatform.MatchingEngineIndex.create_tree_ah_index\u306e\u5f15\u6570\u306e\u30dd\u30a4\u30f3\u30c8\u306fdimensions\u3068approximate_neighbors_count\u306b\u306a\u308a\u307e\u3059\u3002dimensions\u306f\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3057\u305fjson\u306e\u30d9\u30af\u30c8\u30eb\u304c\u4f55\u6b21\u5143\u306e\u3082\u306e\u306a\u306e\u304b\u3092\u5b9a\u7fa9\u3057\u307e\u3059\u3002\u4eca\u56de\u306fOpenAI\u306eEmbeddings API\u306elarge\u30e2\u30c7\u30eb\u3092\u5229\u7528\u3057\u305f\u306e\u3067\u51fa\u529b\u306f3072\u6b21\u5143\u306b\u306a\u308a\u307e\u3059\u3002\u305d\u306e\u305f\u3081\u3001\u3053\u3053\u3067\u3082\u6b21\u5143\u6570\u306f3072\u3068\u5b9a\u7fa9\u3057\u307e\u3059\u3002approximate_neighbors_count\u306f\u985e\u4f3c\u5ea6\u691c\u7d22\u3059\u308b\u969b\u3001\u985e\u4f3c\u3057\u305f\u8981\u7d20\u3092\u4f55\u500b\u51fa\u529b\u3059\u308b\u304b\u3092\u5b9a\u7fa9\u3057\u307e\u3059\u3002\u4eca\u56de\u306f10\u3067\u5b9a\u7fa9\u3057\u3066\u3044\u308b\u306e\u3067\u3001\u985e\u4f3c\u5ea6\u691c\u7d22\u3057\u305f\u7d50\u679c\u306e\u4e2d\u304b\u3089\u985e\u4f3c\u5ea6\u306e\u9ad8\u3044\u8981\u7d20\u309210\u500b\u51fa\u529b\u3059\u308b\u4ed5\u69d8\u306b\u306a\u3063\u3066\u3044\u307e\u3059\u3002\u3053\u3053\u306e\u6570\u5024\u3092\u5909\u3048\u308b\u3053\u3068\u3067\u3001\u51fa\u529b\u6570\u3092\u5236\u5fa1\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<!-- notionvc: 8d468a2e-c051-46be-89e4-f32844a8f71f --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>display_name = f\"&lt; GCP\u306eGUI\u4e0a\u306e\u8868\u8a18\u540d &gt;\" \r\n\r\n# \u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210 \r\nmy_index = aiplatform.MatchingEngineIndex.create_tree_ah_index( \r\n\u3000\u3000display_name = display_name, \r\n\u3000\u3000contents_delta_uri = '&lt; \u5148\u307b\u3069json\u30d5\u30a1\u30a4\u30eb\u3092\u30a2\u30c3\u30d7\u30ed\u30fc\u30c9\u3057\u305fCloud Storage\u306egsutil URI &gt;', \r\n\u3000\u3000dimensions = 3072, # \u30d9\u30af\u30c8\u30eb\u306e\u6b21\u5143\u6570 \r\n\u3000\u3000approximate_neighbors_count = 10)\u3000# \u985e\u4f3c\u5ea6\u691c\u7d22\u306e\u51fa\u529b\u6570 <\/code><\/pre>\r\n<\/div>\r\n<p>\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210\u304c\u3067\u304d\u305f\u3089\u3001\u6b21\u306f\u30a8\u30f3\u30c9\u30dd\u30a4\u30f3\u30c8\u3092\u4f5c\u6210\u3057\u30c7\u30d7\u30ed\u30a4\u3057\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3057\u3001\u30a8\u30f3\u30c9\u30dd\u30a4\u30f3\u30c8\u306e\u4f5c\u6210\u3068\u30c7\u30d7\u30ed\u30a4\u3092\u884c\u306a\u3063\u3066\u4e0b\u3055\u3044\uff08\u5c11\u3057\u6642\u9593\u304c\u304b\u304b\u308a\u307e\u3059\uff09\u3002\u3053\u306e\u5b9f\u884c\u304c\u5b8c\u4e86\u3059\u308b\u3068\u3001\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u304c\u5229\u7528\u3067\u304d\u308b\u72b6\u614b\u306b\u306a\u308a\u307e\u3059\u3002<!-- notionvc: 3e1a92dd-2869-4d0e-8db7-b1651163c38c --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code># IndexEndpoint\u306e\u4f5c\u6210 \r\nmy_index_endpoint = aiplatform.MatchingEngineIndexEndpoint.create( \r\n\u3000\u3000display_name = display_name, \r\n\u3000\u3000public_endpoint_enabled = True \r\n) \r\n\r\n# Index Endpoint\u306e\u30c7\u30d7\u30ed\u30a4 \r\nmy_index_endpoint.deploy_index( \r\n\u3000\u3000index = my_index, \r\n\u3000\u3000deployed_index_id = display_name \r\n)<!-- notionvc: cf2dad10-7865-4e71-80cf-8b4d9022247f --><\/code><\/pre>\r\n<\/div>\n\n<h2>\u5b9f\u88c5\u30d5\u30ed\u30fc\u2464 \u301cLangChain\u306e\u5b9f\u88c5\u301c<\/h2>\n<p>\u30c6\u30ad\u30b9\u30c8\u306e\u30d9\u30af\u30c8\u30eb\u5316\u3068\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210\u304c\u6e08\u3093\u3060\u306e\u3067\u3044\u3088\u3044\u3088LangChain\u90e8\u5206\u306e\u5b9f\u88c5\u3092\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u307e\u305a\u3001\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3057\u5fc5\u8981\u306a\u30e9\u30a4\u30d6\u30e9\u30ea\u3092\u30a4\u30f3\u30dd\u30fc\u30c8\u3057\u3066\u4e0b\u3055\u3044\u3002<!-- notionvc: 6097b5f5-4bfb-4d1e-a5f2-8f7a5fca6e3b --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>import os \r\nfrom langchain_openai import ChatOpenAI \r\nfrom langchain_core.prompts import ChatPromptTemplate \r\nfrom langchain.chains.combine_documents import create_stuff_documents_chain \r\nfrom langchain_core.documents import Document \r\nfrom google.cloud import aiplatform \r\nfrom google.cloud import translate \r\nimport json<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u3001\u5148\u307b\u3069\u30c7\u30d7\u30ed\u30a4\u3057\u305f\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u30a8\u30f3\u30c9\u30dd\u30a4\u30f3\u30c8\u3092\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3057\u3066\u30a8\u30f3\u30c9\u30dd\u30a4\u30f3\u30c8\u3092\u547c\u3073\u51fa\u3057\u3066\u4e0b\u3055\u3044\u3002my_index_endpoint_id\u306fGCP\u306e\u30b3\u30f3\u30bd\u30fc\u30eb\u753b\u9762\u3067\u78ba\u8a8d\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<!-- notionvc: fe1a0c9e-94a2-4418-baf8-2b7d0c34e7cc --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>def load_index_endpoint(): \r\n\u3000\u3000# \u5148\u8ff0\u306e\u95a2\u6570\u3092\u5229\u7528\u3057\u3066\u3044\u307e\u3059\u3002 \r\n\u3000\u3000project_id, location, credentials = init_gcp() \r\n\r\n\u3000\u3000# AI platform\u306e\u521d\u671f\u5316 \r\n\u3000\u3000aiplatform.init(project=project_id, location=location, credentials=credentials) \r\n\u3000\u3000my_index_endpoint_id = \"&lt; \u30c7\u30d7\u30ed\u30a4\u3057\u305f\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u30dd\u30a4\u30f3\u30c8\u306eid &gt;\" \r\n\u3000\u3000my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(my_index_endpoint_id) \r\n\u3000\u3000return my_index_endpoint \r\n\r\n# \u30a8\u30f3\u30c9\u30dd\u30a4\u30f3\u30c8\u306e\u547c\u3073\u51fa\u3057 \r\nmy_index_endpoint = load_index_endpoint()<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u3001\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u306e\u7ffb\u8a33\u51e6\u7406\u3092\u5b9f\u88c5\u3057\u307e\u3059\u3002\u4eca\u56de\u30d9\u30af\u30c8\u30eb\u5316\u3057\u305f\u5916\u90e8\u30c7\u30fc\u30bf\u306f\u82f1\u8a9e\u306e\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u306e\u305f\u3081\u3001\u65e5\u672c\u8a9e\u3088\u308a\u3082\u82f1\u8a9e\u306e\u65b9\u304c\u985e\u4f3c\u5ea6\u691c\u7d22\u304c\u4e0a\u624b\u304f\u6a5f\u80fd\u3059\u308b\u3068\u8003\u3048\u3053\u306e\u3088\u3046\u306a\u4ed5\u69d8\u306b\u3057\u307e\u3057\u305f\u3002\u307e\u305f\u3001\u4eca\u56de\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u3059\u308b\u306e\u306f\u300cAWS\u306eCUR\u306b\u95a2\u3059\u308b\u8cea\u554f\u3067\u3059\u3002\u4fa1\u683c\u3092\u610f\u5473\u3059\u308b\u30c7\u30fc\u30bf\u5217\u306bBlendedCost\u3068unBlendedCost\u304c\u3042\u308b\u3068\u601d\u3044\u307e\u3059\u304c\u3001\u3053\u308c\u3089\u306f\u3069\u306e\u3088\u3046\u306a\u9055\u3044\u304c\u3042\u308b\u306e\u3067\u3057\u3087\u3046\u304b\uff1f\u300d\u306b\u3057\u307e\u3059\u3002<\/p>\r\n<p>\u672c\u95a2\u6570\u306e\u30dd\u30a4\u30f3\u30c8\u306fsource_language_code\u3068target_language_code\u3067\u3059\u3002\u3053\u308c\u3089\u306f\u3001\u7ffb\u8a33\u3057\u305f\u3044\u30c6\u30ad\u30b9\u30c8\u306e\u73fe\u8a00\u8a9e\u3068\u3001\u3069\u306e\u8a00\u8a9e\u306b\u7ffb\u8a33\u3057\u305f\u3044\u306e\u304b\u3092\u5b9a\u7fa9\u3057\u307e\u3059\u3002\u4eca\u56de\u306f\u65e5\u672c\u8a9e\u306e\u30d7\u30ed\u30f3\u30d7\u30c8\u3092\u82f1\u8a9e\u306b\u7ffb\u8a33\u3057\u305f\u3044\u306e\u3067\u3001source_language_code\u3092ja\u3001target_language_code\u3092en-US\u3068\u5b9a\u7fa9\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\r\n<p><!-- notionvc: d2cbb08f-6168-47f6-8128-1990cd1ec209 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>query = 'AWS\u306eCUR\u306b\u95a2\u3059\u308b\u8cea\u554f\u3067\u3059\u3002\u4fa1\u683c\u3092\u610f\u5473\u3059\u308b\u30c7\u30fc\u30bf\u5217\u306bBlendedCost\u3068unBlendedCost\u304c\u3042\u308b\u3068\u601d\u3044\u307e\u3059\u304c\u3001\u3053\u308c\u3089\u306f\u3069\u306e\u3088\u3046\u306a\u9055\u3044\u304c\u3042\u308b\u306e\u3067\u3057\u3087\u3046\u304b\uff1f' \r\n\r\ndef text_translate(text, source_language_code, target_language_code): \r\n\u3000\u3000# \u5148\u8ff0\u306e\u95a2\u6570\u3092\u4f7f\u7528 \r\n\u3000\u3000project_id, _, credentials = init_gcp() \r\n\u3000\u3000client = translate.TranslationServiceClient(credentials=credentials) \r\n\u3000\u3000location = \"global\" \r\n\u3000\u3000parent = f\"projects\/{project_id}\/locations\/{location}\" \r\n\u3000\u3000response = client.translate_text( \r\n\u3000\u3000\u3000\u3000request={ \r\n\u3000\u3000\u3000\u3000\u3000\u3000\"parent\": parent, \r\n\u3000\u3000\u3000\u3000\u3000\u3000\"contents\": [text], \r\n\u3000\u3000\u3000\u3000\u3000\u3000\"mime_type\": \"text\/plain\", \r\n\u3000\u3000\u3000\u3000\u3000\u3000\"source_language_code\": source_language_code, \r\n\u3000\u3000\u3000\u3000\u3000\u3000\"target_language_code\": target_language_code \r\n\u3000\u3000\u3000\u3000} \r\n\u3000\u3000) \r\n\u3000\u3000return response.translations[0].translated_text \r\n\r\nquery = text_translate(query, 'ja', 'en-US')<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u3001\u82f1\u8a33\u3057\u305f\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3067\u5916\u90e8\u30c7\u30fc\u30bf\u306b\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u3059\u308b\u305f\u3081\u306b\u3001\u82f1\u8a33\u3057\u305f\u30d7\u30ed\u30f3\u30d7\u30c8\u3092\u30d9\u30af\u30c8\u30eb\u5316\u3057\u307e\u3059\u3002\u4e0b\u8a18\u3092\u5b9f\u884c\u3057\u3066\u3001\u30d7\u30ed\u30f3\u30d7\u30c8\u3092\u30d9\u30af\u30c8\u30eb\u5316\u3057\u307e\u3059\u3002\u3053\u306e\u90e8\u5206\u306e\u4ed5\u69d8\u306f\u4ee5\u524d\u5916\u90e8\u30c7\u30fc\u30bf\u3092\u30d9\u30af\u30c8\u30eb\u5316\u3057\u305f\u6642\u3068\u540c\u3058\u3067\u3059\u3002<!-- notionvc: 83caea86-3343-4dc8-a15f-fb01ecd3ec39 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code># API\u30ad\u30fc\u3092\u8a2d\u5b9a \u3000\r\napi_key = '&lt; OpenAI\u306eAPI\u30ad\u30fc &gt;' \r\n\r\ndef convert_query_vector(api_key, query): \r\n\u3000\u3000os.environ[\"OPENAI_API_KEY\"] = api_key \r\n\u3000\u3000openai_client = OpenAI() \r\n\u3000\u3000query_vector = openai_client.embeddings.create(model=\"text-embedding-3-large\", input=query).data[0].embedding \r\n\u3000\u3000return query_vector \r\n\r\nquery_vector = convert_query_vector(api_key, query)<\/code><\/pre>\r\n<\/div>\r\n<p>\u6b21\u306b\u3001\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u985e\u4f3c\u3057\u3066\u3044\u308b\u30c6\u30ad\u30b9\u30c8\u306eid\u3092\u53d6\u5f97\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3057\u3001\u5165\u529b\u30dd\u30a4\u30f3\u30c8\u3068\u985e\u4f3c\u5ea6\u306e\u9ad8\u3044\u4e0a\u4f4d10\u500b\u306e\u30c6\u30ad\u30b9\u30c8\u306eid\u3092\u53d6\u5f97\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u307e\u3059\u3002<!-- notionvc: 473b4f9f-1580-452f-9bc9-e09fc8cf92b7 --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>deployed_index_id = f\" \u30c7\u30d7\u30ed\u30a4\u3057\u305f\u30a8\u30f3\u30c9\u30dd\u30a4\u30f3\u30c8\u306e\u8868\u8a18\u540d \" \r\n\r\n# \u30d9\u30af\u30c8\u30eb\u691c\u7d22\u306e\u7d50\u679c\u304c\u8fd4\u3063\u3066\u304f\u308b \r\nresponse = my_index_endpoint.find_neighbors( \r\n\u3000\u3000deployed_index_id = deployed_index_id, \r\n\u3000\u3000queries = [query_vector], \r\n\u3000\u3000num_neighbors = 10 \r\n) \r\n\r\n# \u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u985e\u4f3c\u5ea6\u306e\u9ad8\u3044\u30c6\u30ad\u30b9\u30c8\u306eid\u3092id_list\u306b\u683c\u7d0d\u3059\u308b \r\nid_list = () \r\nfor id, neighbor in enumerate(response[0]): \r\n\u3000\u3000id_list = (id_list +(neighbor.id,))<\/code><\/pre>\r\n<\/div>\r\n<p>\u4e0a\u8a18\u3067\u306f\u3001\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u985e\u4f3c\u5ea6\u306e\u9ad8\u3044\u30c6\u30ad\u30b9\u30c8\u306eid\u306f\u53d6\u5f97\u3067\u304d\u307e\u3059\u304c\u3001\u305d\u306e\u30c6\u30ad\u30b9\u30c8\u304c\u3069\u3093\u306a\u30c6\u30ad\u30b9\u30c8\u306a\u306e\u304b\u306f\u53d6\u5f97\u3067\u304d\u307e\u305b\u3093\u3002\u305d\u306e\u305f\u3081\u3001\u5148\u307b\u3069Bigquery\u306b\u4fdd\u5b58\u3057\u305f\u30c6\u30fc\u30d6\u30eb\u304b\u3089id_list\u306e\u30c6\u30ad\u30b9\u30c8\u3076\u3093\u3092\u53d6\u5f97\u3057\u3066\u304d\u307e\u3059\u3002\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3057\u30c6\u30ad\u30b9\u30c8\u3092\u53d6\u5f97\u3057\u3066\u3044\u304d\u307e\u3059\u3002<!-- notionvc: e204f926-8005-4e76-9b3c-6126965c6bfd --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code>bq_query = f\"\"\" SELECT * FROM &lt; \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u4fdd\u5b58\u3057\u305f\u30c6\u30fc\u30d6\u30eb\u306e\u30d1\u30b9 &gt; WHERE ID IN {id_list} \"\"\" \r\n\r\ndef load_data(query): \r\n\u3000\u3000_, credentials = load_credentials() \r\n\u3000\u3000bq_client = bigquery.Client(credentials=credentials) \r\n\u3000\u3000return bq_client.query(query).to_dataframe() \r\n\r\ndf = load_data(bq_query) \r\n\r\n# \u53d6\u5f97\u3057\u3066\u304d\u305f\u30c6\u30ad\u30b9\u30c8\u3092list\u5316 \r\ncontext = list(df.cur_guide_text) \r\n\r\n# LangChain\u306b\u5411\u3051\u3066\u300110\u500b\u306e\u30c6\u30ad\u30b9\u30c8\u6587\u3092\uff11\u3064\u306e\u6587\u5b57\u5217\u306b\u5909\u63db \r\ncontext = [sentence + \".\" if not sentence.endswith(\".\") else sentence for sentence in context] \r\ncontext = \" \".join(context)<\/code><\/pre>\r\n<\/div>\r\n<p>\u3044\u3088\u3044\u3088LangChain\u306e\u30b3\u30a2\u90e8\u5206\u306e\u5b9f\u88c5\u3092\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u5927\u307e\u304b\u306a\u6d41\u308c\u306f\u4e0b\u8a18\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<!-- notionvc: 1c13e9e3-2514-43e8-8dd3-de72bbf2d6af --><\/p>\r\n<p><img decoding=\"async\" src=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2024\/02\/langchain_flow.png\" \/><\/p>\r\n<p>\u4e0b\u8a18\u30b3\u30fc\u30c9\u3092\u5b9f\u884c\u3059\u308b\u3053\u3068\u3067\u3001RAG\u306b\u3088\u308b\u6587\u7ae0\u751f\u6210\u304c\u53ef\u80fd\u306b\u306a\u308a\u307e\u3059\u3002<!-- notionvc: d7534422-158f-45ab-8ebe-f8cb21a70fec --><\/p>\r\n<div class=\"hcb_wrap\">\r\n<pre class=\"prism line-numbers lang-python\" data-lang=\"Python\"><code><\/code># \u4f7f\u7528\u3059\u308bLLM\u30e2\u30c7\u30eb\u3092\u5b9a\u7fa9 \u3000 \r\ngpt_model = 'gpt-4-0125-preview' \r\n\r\n# LangChain\u7528\u306e\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306e\u4f5c\u6210 \r\nllm = ChatOpenAI(model_name=gpt_model, openai_api_key=api_key) \r\n\r\n# \u30d7\u30ed\u30f3\u30d7\u30c8\u306e\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u3092\u5b9a\u7fa9 \r\nprompt = ChatPromptTemplate.from_template(\"\"\"Answer the following question based only on the provided \u3000\u3000\u3000\u3000\r\ncontext: <context>{context} <\/context> \u3000\u3000\u3000\u3000\r\nQuestion: {input}\r\n\"\"\" ) \r\n\r\n# LLM\u3068\u30d7\u30ed\u30f3\u30d7\u30c8\u3092chain \r\ndocument_chain = create_stuff_documents_chain(llm, prompt) \r\n\r\n# \u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3068\u985e\u4f3c\u5ea6\u306e\u9ad8\u304b\u3063\u305f\u30c6\u30ad\u30b9\u30c8\u30c7\u30fc\u30bf\u3092\u5148\u307b\u3069\u306e\u30d7\u30ed\u30f3\u30d7\u30c8\u306e\u30d5\u30a9\u30fc\u30de\u30c3\u30c8\u306b\u57cb\u3081\u8fbc\u307f\u5b9f\u884c \r\nresponse_rag = document_chain.invoke({ \u3000\u3000\r\n\"input\": query, # \u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8 \u3000\u3000\r\n\"context\": [Document(page_content=context)] # \u985e\u4f3c\u5ea6\u306e\u9ad8\u304b\u3063\u305f\u30c6\u30ad\u30b9\u30c8\u30c7\u30fc\u30bf\u00a0 \u3000\u3000 \r\n} ) \r\n\r\n# \u8fd4\u3063\u3066\u304d\u305f\u51fa\u529b\u3092\u82f1\u8a9e\u304b\u3089\u65e5\u672c\u8a9e\u306b\u7ffb\u8a33 \r\nresponse_rag = text_translate(response_rag, 'en-US', 'ja')<!-- notionvc: 94970a09-59d5-437d-b210-b89b3e0db7df --><\/pre>\r\n<\/div>\n\n<h2>\u691c\u8a3c<\/h2>\n<p>\u4eca\u56de\u306e\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\u3092RAG\u3042\u308a\u30d0\u30fc\u30b8\u30e7\u30f3\u3068RAG\u306a\u3057\u30d0\u30fc\u30b8\u30e7\u30f3\u3067\u51fa\u529b\u3057\u307e\u3057\u305f\u3002<\/p>\r\n<p>\u5165\u529b\u30d7\u30ed\u30f3\u30d7\u30c8\uff1aAWS\u306eCUR\u306b\u95a2\u3059\u308b\u8cea\u554f\u3067\u3059\u3002\u4fa1\u683c\u3092\u610f\u5473\u3059\u308b\u30c7\u30fc\u30bf\u5217\u306bBlendedCost\u3068unBlendedCost\u304c\u3042\u308b\u3068\u601d\u3044\u307e\u3059\u304c\u3001\u3053\u308c\u3089\u306f\u3069\u306e\u3088\u3046\u306a\u9055\u3044\u304c\u3042\u308b\u306e\u3067\u3057\u3087\u3046\u304b\uff1f<\/p>\r\n<p><!-- notionvc: 71867ef6-bb19-4fd1-85cf-ba4fb233205e --><\/p>\r\n<h1>RAG\u3042\u308a\u30d0\u30fc\u30b8\u30e7\u30f3\u306e\u51fa\u529b<\/h1>\r\n<p>AWS CUR (\u30b3\u30b9\u30c8\u3068\u4f7f\u7528\u72b6\u6cc1\u30ec\u30dd\u30fc\u30c8) \u3067\u306f\u3001BlendedCost \u3068 unBlendedCost \u306e\u9055\u3044\u306f\u3001\u4e00\u62ec\u8acb\u6c42\u3092\u4f7f\u7528\u3057\u3066 AWS Organizations \u5185\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u306e\u30b3\u30b9\u30c8\u304c\u8a08\u7b97\u3055\u308c\u308b\u65b9\u6cd5\u306b\u95a2\u4fc2\u3057\u307e\u3059\u3002<\/p>\r\n<p><strong>BlendedCost<\/strong>: \u3053\u308c\u306f\u3001AWS \u7d44\u7e54\u5185\u306e\u3059\u3079\u3066\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u3067\u4f7f\u7528\u3055\u308c\u308b AWS \u30b5\u30fc\u30d3\u30b9\u306e\u5e73\u5747\u30b3\u30b9\u30c8\u3092\u8868\u3057\u307e\u3059\u3002\u4e00\u62ec\u8acb\u6c42\u3092\u4f7f\u7528\u3059\u308b\u5834\u5408\u3001AWS \u306f\u3059\u3079\u3066\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u306b\u308f\u305f\u308b\u30b5\u30fc\u30d3\u30b9 (EC2 \u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306a\u3069) \u306e\u30b3\u30b9\u30c8\u3092\u5e73\u5747\u3057\u3066 BlendedCost \u3092\u8a08\u7b97\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u7d44\u7e54\u5185\u306e\u7570\u306a\u308b\u30a2\u30ab\u30a6\u30f3\u30c8\u304c\u540c\u3058\u30b5\u30fc\u30d3\u30b9\u306b\u5bfe\u3057\u3066\u7570\u306a\u308b\u6599\u91d1\u3092\u652f\u6255\u3063\u3066\u3044\u308b\u5834\u5408 (\u30dc\u30ea\u30e5\u30fc\u30e0 \u30c7\u30a3\u30b9\u30ab\u30a6\u30f3\u30c8\u3001\u4e88\u7d04\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306a\u3069\u306b\u3088\u308a)\u3001\u6df7\u5408\u30b3\u30b9\u30c8\u306b\u306f\u3001\u3059\u3079\u3066\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u306b\u308f\u305f\u308b\u305d\u306e\u30b5\u30fc\u30d3\u30b9\u306e\u5e73\u5747\u6599\u91d1\u304c\u53cd\u6620\u3055\u308c\u308b\u3053\u3068\u3092\u610f\u5473\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u7d44\u7e54\u306e AWS \u4f7f\u7528\u306e\u5168\u4f53\u7684\u306a\u8cbb\u7528\u5bfe\u52b9\u679c\u3092\u7406\u89e3\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\r\n<p><strong>unBlendedCost<\/strong>: \u5bfe\u7167\u7684\u306b\u3001unBlendedCost \u306f\u3001\u7d44\u7e54\u5168\u4f53\u3067\u5e73\u5747\u3059\u308b\u3053\u3068\u306a\u304f\u3001\u5404\u500b\u5225\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u3067\u4f7f\u7528\u3055\u308c\u308b AWS \u30b5\u30fc\u30d3\u30b9\u306e\u5b9f\u969b\u306e\u30b3\u30b9\u30c8\u3092\u6307\u3057\u307e\u3059\u3002\u3053\u306e\u30b3\u30b9\u30c8\u306b\u306f\u3001\u5404\u30a2\u30ab\u30a6\u30f3\u30c8\u306b\u9069\u7528\u3055\u308c\u308b\u7279\u5b9a\u306e\u6599\u91d1\u3068\u5272\u5f15\u304c\u53cd\u6620\u3055\u308c\u3066\u304a\u308a\u3001AWS \u30b5\u30fc\u30d3\u30b9\u306b\u5bfe\u3057\u3066\u305d\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u306b\u5b9f\u969b\u306b\u8acb\u6c42\u3055\u308c\u3066\u3044\u308b\u91d1\u984d\u3092\u3088\u308a\u6b63\u78ba\u306b\u628a\u63e1\u3067\u304d\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u6df7\u5408\u30b3\u30b9\u30c8\u306b\u898b\u3089\u308c\u308b\u5e73\u5747\u5316\u52b9\u679c\u306a\u3057\u3067\u3001\u7279\u5b9a\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u306e\u4f7f\u7528\u306b\u95a2\u9023\u3059\u308b\u76f4\u63a5\u30b3\u30b9\u30c8\u3092\u78ba\u8a8d\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\r\n<p>\u3053\u308c\u3089 2 \u7a2e\u985e\u306e\u30b3\u30b9\u30c8\u306e\u9055\u3044\u3092\u7406\u89e3\u3059\u308b\u3053\u3068\u306f\u3001\u7d44\u7e54\u304c AWS \u652f\u51fa\u3092\u3088\u308a\u52b9\u679c\u7684\u306b\u5206\u6790\u3057\u3001\u30b3\u30b9\u30c8\u304c\u767a\u751f\u3057\u3066\u3044\u308b\u5834\u6240\u3068\u3001\u30b3\u30b9\u30c8\u3092\u7bc0\u7d04\u3059\u308b\u305f\u3081\u306b AWS \u306e\u4f7f\u7528\u3092\u6700\u9069\u5316\u3059\u308b\u65b9\u6cd5\u3092\u7279\u5b9a\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\r\n<p><!-- notionvc: ff1f7015-dd96-4758-9519-74fc403d5bba --><\/p>\r\n<h1>RAG\u306a\u3057\u30d0\u30fc\u30b8\u30e7\u30f3\u306e\u51fa\u529b<!-- notionvc: 420e0122-9954-43bf-8b94-ff93ecddc637 --><\/h1>\r\n<p>AWS \u306e\u30b3\u30b9\u30c8\u3068\u4f7f\u7528\u72b6\u6cc1\u30ec\u30dd\u30fc\u30c8 (CUR) \u306f\u3001AWS \u30b5\u30fc\u30d3\u30b9\u306e\u4f7f\u7528\u3068\u305d\u306e\u30b3\u30b9\u30c8\u306b\u95a2\u3059\u308b\u8a73\u7d30\u306a\u30c7\u30fc\u30bf\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002 AWS \u306e\u8acb\u6c42\u66f8\u307e\u305f\u306f CUR \u3092\u5206\u6790\u3059\u308b\u3068\u304d\u3001\u300cBlendedCost\u300d\u3084\u300cUnblendedCost\u300d\u306a\u3069\u306e\u3055\u307e\u3056\u307e\u306a\u30b3\u30b9\u30c8\u95a2\u9023\u306e\u7528\u8a9e\u304c\u767b\u5834\u3057\u307e\u3059\u3002 AWS \u306e\u652f\u51fa\u3092\u6b63\u78ba\u306b\u5206\u6790\u3059\u308b\u306b\u306f\u3001\u3053\u308c\u3089\u306e\u7528\u8a9e\u306e\u9055\u3044\u3092\u7406\u89e3\u3059\u308b\u3053\u3068\u304c\u91cd\u8981\u3067\u3059\u3002<\/p>\r\n<b>UnblendedCost<\/b>\r\n<p><strong>\u5b9a\u7fa9<\/strong>: UnblendedCost \u306f\u3001\u5272\u5f15\u3001\u30ea\u30b6\u30fc\u30d6\u30c9\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3001\u307e\u305f\u306f\u7bc0\u7d04\u30d7\u30e9\u30f3\u3092\u8003\u616e\u305b\u305a\u306b\u3001\u5404 AWS \u30b5\u30fc\u30d3\u30b9\u306e\u4f7f\u7528\u306b\u304b\u304b\u308b\u76f4\u63a5\u30b3\u30b9\u30c8\u3092\u8868\u3057\u307e\u3059\u3002\u57fa\u672c\u7684\u306b\u306f\u3001\u300c\u5b9a\u4fa1\u300d\u306b\u4f7f\u7528\u91cf\u3092\u4e57\u3058\u305f\u91d1\u984d\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\r\n<p><strong>\u4f7f\u7528\u4f8b<\/strong>: \u4fa1\u683c\u30e2\u30c7\u30eb\u3084\u5272\u5f15\u9069\u7528\u306e\u5f71\u97ff\u3092\u53d7\u3051\u305a\u306b\u3001AWS \u30ea\u30bd\u30fc\u30b9\u306b\u95a2\u3057\u3066\u3069\u308c\u3060\u3051\u6d88\u8cbb\u3057\u3066\u3044\u308b\u304b\u3092\u628a\u63e1\u3059\u308b\u306e\u306b\u7279\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002\u3053\u306e\u5024\u306f\u3001\u5272\u5f15\u3084\u7279\u5225\u306a\u4fa1\u683c\u8abf\u6574\u304c\u9069\u7528\u3055\u308c\u308b\u524d\u306b\u3001\u5b9f\u969b\u306e\u4f7f\u7528\u30b3\u30b9\u30c8\u3092\u8a55\u4fa1\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\r\n<b>blendedCost<\/b>\r\n<p><strong>\u5b9a\u7fa9<\/strong>:<\/p>\r\n<p>\u4e00\u65b9\u3001BlendedCost \u3067\u306f\u3001\u4e00\u62ec\u8acb\u6c42\u30d5\u30a1\u30df\u30ea\u30fc\u5185\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u9593\u306e\u5272\u5f15\u3068\u4fa1\u683c\u306e\u5e73\u5747\u5316\u304c\u8003\u616e\u3055\u308c\u307e\u3059\u3002\u30ea\u30b6\u30fc\u30d6\u30c9\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9 (RI) \u307e\u305f\u306f\u7bc0\u7d04\u30d7\u30e9\u30f3\u3092\u4f7f\u7528\u3057\u3066\u3044\u308b\u7d44\u7e54\u306e\u5834\u5408\u3001AWS \u306f\u6df7\u5408\u6599\u91d1\u3092\u9069\u7528\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u7d44\u7e54\u5185\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u5168\u4f53\u3067\u30ea\u30b6\u30fc\u30d6\u30c9\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3068\u30aa\u30f3\u30c7\u30de\u30f3\u30c9\u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u306e\u30b3\u30b9\u30c8\u3092\u5e73\u5747\u3057\u307e\u3059\u3002<\/p>\r\n<p><strong>\u4f7f\u7528\u4f8b<\/strong>: \u3053\u308c\u306f\u3001\u4e00\u62ec\u8acb\u6c42\u30d5\u30a1\u30df\u30ea\u30fc\u306e\u3059\u3079\u3066\u306e\u30a2\u30ab\u30a6\u30f3\u30c8\u306b\u308f\u305f\u308b\u30ea\u30b6\u30fc\u30d6\u30c9 \u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3068\u7bc0\u7d04\u30d7\u30e9\u30f3\u306e\u30b3\u30b9\u30c8\u52b9\u7387\u3068\u5f71\u97ff\u3092\u5206\u6790\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002\u3053\u308c\u306f\u3001AWS \u652f\u51fa\u306e\u5168\u4f53\u7684\u306a\u8cbb\u7528\u5bfe\u52b9\u679c\u3092\u7406\u89e3\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\r\n<b>\u4e3b\u306a\u9055\u3044:<\/b>\r\n<p><strong>\u4fa1\u683c\u306b\u95a2\u3059\u308b\u8003\u616e\u4e8b\u9805<\/strong>: BlendedCost \u3067\u306f\u3001\u7d71\u5408\u8acb\u6c42\u5148\u30a2\u30ab\u30a6\u30f3\u30c8\u5168\u4f53\u3067\u306e\u30ea\u30b6\u30fc\u30d6\u30c9 \u30a4\u30f3\u30b9\u30bf\u30f3\u30b9\u3001\u7bc0\u7d04\u30d7\u30e9\u30f3\u3001\u305d\u306e\u4ed6\u306e\u5272\u5f15\u30e1\u30ab\u30cb\u30ba\u30e0\u306e\u5f71\u97ff\u304c\u8003\u616e\u3055\u308c\u307e\u3059\u304c\u3001UnblendedCost \u3067\u306f\u3001\u305d\u306e\u3088\u3046\u306a\u8003\u616e\u4e8b\u9805\u304c\u8003\u616e\u3055\u308c\u3066\u3044\u306a\u3044\u30b3\u30b9\u30c8\u304c\u8868\u793a\u3055\u308c\u307e\u3059\u3002<\/p>\r\n<p><strong>\u8acb\u6c42\u306e\u6700\u9069\u5316\u3067\u306e\u4f7f\u7528<\/strong>: UnblendedCost \u306f\u3001\u30b5\u30fc\u30d3\u30b9\u306e\u5b9f\u969b\u306e\u4f7f\u7528\u30b3\u30b9\u30c8\u3092\u7279\u5b9a\u3059\u308b\u305f\u3081\u306b\u91cd\u8981\u3067\u3059\u3002\u3053\u308c\u306f\u3001\u5272\u5f15\u3092\u9069\u7528\u3059\u308b\u524d\u306b\u3001\u6700\u9069\u5316\u307e\u305f\u306f\u30b3\u30b9\u30c8\u524a\u6e1b\u306e\u53ef\u80fd\u6027\u306e\u3042\u308b\u9818\u57df\u3092\u7279\u5b9a\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002 BlendedCost \u306f\u3001\u7d44\u7e54\u5168\u4f53\u3067\u306e RI \u3084\u8caf\u84c4\u30d7\u30e9\u30f3\u306e\u4f7f\u7528\u72b6\u6cc1\u306a\u3069\u3001\u30b3\u30b9\u30c8\u6700\u9069\u5316\u6226\u7565\u306e\u5168\u4f53\u7684\u306a\u6709\u52b9\u6027\u3092\u8a55\u4fa1\u3059\u308b\u306e\u306b\u5f79\u7acb\u3061\u307e\u3059\u3002<\/p>\r\n<p><strong>\u30b3\u30b9\u30c8\u5206\u6790\u3078\u306e\u52b9\u679c<\/strong>: \u591a\u304f\u306e RI \u307e\u305f\u306f\u7bc0\u7d04\u30d7\u30e9\u30f3\u3092\u4f7f\u7528\u3057\u3066\u3044\u308b\u7d44\u7e54\u306e\u5834\u5408\u3001BlendedCost \u306f\u3001\u30b3\u30b9\u30c8\u5171\u6709\u6226\u7565\u306e\u5229\u70b9\u3092\u542b\u3081\u3001\u7d44\u7e54\u5185\u306e\u5168\u54e1\u304c\u8ca2\u732e\u3057\u3066\u3044\u308b\u5b9f\u969b\u306e\u30b3\u30b9\u30c8\u306e\u3088\u308a\u6b63\u78ba\u306a\u60c5\u5831\u3092\u63d0\u4f9b\u3057\u307e\u3059\u3002\u305f\u3060\u3057\u3001\u8a73\u7d30\u306a\u30b5\u30fc\u30d3\u30b9 \u30ec\u30d9\u30eb\u5206\u6790\u306e\u5834\u5408\u3001\u307e\u305f\u306f\u65b0\u3057\u3044\u4e88\u7d04\u3084\u5272\u5f15\u30d7\u30e9\u30f3\u3092\u691c\u8a0e\u3059\u308b\u5834\u5408\u306f\u3001UnblendedCosts \u3092\u8abf\u3079\u3066\u3001\u3055\u3089\u306a\u308b\u5272\u5f15\u306e\u6a5f\u4f1a\u3092\u7406\u89e3\u3059\u308b\u3053\u3068\u3082\u3067\u304d\u307e\u3059\u3002<\/p>\r\n<p>\u8981\u7d04\u3059\u308b\u3068\u3001BlendedCost \u3068 UnblendedCost \u306e\u4e21\u65b9\u304c AWS \u306e\u30b3\u30b9\u30c8\u7ba1\u7406\u3068\u6700\u9069\u5316\u306b\u304a\u3044\u3066\u5f79\u5272\u3092\u679c\u305f\u3057\u307e\u3059\u3002\u3069\u3061\u3089\u3092\u4f7f\u7528\u3059\u308b\u304b\u3092\u9078\u629e\u3059\u308b\u306e\u306f\u3001\u5b9f\u969b\u306e\u4f7f\u7528\u30b3\u30b9\u30c8\u3092\u5206\u6790\u3059\u308b\u304b\u3001\u652f\u51fa\u3092\u6700\u9069\u5316\u3059\u308b\u304b\u3001\u4e00\u62ec\u8acb\u6c42\u30d5\u30a1\u30df\u30ea\u30fc\u5168\u4f53\u3067\u306e\u7bc0\u7d04\u6226\u7565\u306e\u6709\u52b9\u6027\u3092\u8a55\u4fa1\u3059\u308b\u304b\u306a\u3069\u3001\u7279\u5b9a\u306e\u30cb\u30fc\u30ba\u306b\u3088\u3063\u3066\u7570\u306a\u308a\u307e\u3059\u3002<\/p>\r\n<p><!-- notionvc: 7b3b587d-3631-459d-b8d8-c9aee4c7e365 --><\/p>\r\n<h1>\u51fa\u529b\u7d50\u679c\u306e\u30ec\u30d3\u30e5\u30fc<!-- notionvc: 5c7156cf-cf16-498b-bd17-fe23850d3f76 --><\/h1>\r\n<p>\u4e21\u8005\u3068\u3082\u306b\u3001\u8cea\u554f\u306b\u5bfe\u3057\u3066\u3057\u3063\u304b\u308a\u56de\u7b54\u306f\u3067\u304d\u3066\u3044\u307e\u3059\u3002\u305f\u3060\u3001RAG\u3042\u308a\u306e\u65b9\u304c\u4e0b\u8a18\u89b3\u70b9\u3067\u51fa\u529b\u7cbe\u5ea6\u304c\u9ad8\u3044\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u8003\u3048\u3066\u3044\u307e\u3059\u3002<\/p>\r\n<ul>\r\n\t<li>RAG\u306a\u3057\u306e\u5834\u5408\u306f\u3001\u8cea\u554f\u306b\u5bfe\u3057\u3066\u4f59\u5206\u306a\u8a00\u53ca\u304c\u591a\u304f\u898b\u53d7\u3051\u3089\u308c\u305f\u3002\u4e00\u65b9RAG\u3042\u308a\u306e\u5834\u5408\u306f\u3001\u8cea\u554f\u306b\u5bfe\u3057\u3066\u30c0\u30a4\u30ec\u30af\u30c8\u306a\u56de\u7b54\u306e\u307f\u3092\u51fa\u529b\u3057\u3066\u3044\u308b\u305f\u3081\u3001\u8cea\u554f\u306b\u5bfe\u3057\u3066\u306e\u56de\u7b54\u306e\u8981\u70b9\u304c\u307e\u3068\u307e\u3063\u3066\u3044\u308b\u3002<\/li>\r\n\t<li>RAG\u306a\u3057\u306e\u5834\u5408\u306f\u3001\u6587\u8a00\u304c\u5b9a\u7fa9\u4ed8\u3051\u3089\u308c\u305f\u6587\u7ae0\u611f\u304c\u3042\u308a\u3001\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u306b\u8a18\u8f09\u3055\u308c\u3066\u3044\u308b\u6587\u7ae0\u3092\u305d\u306e\u307e\u307e\u51fa\u529b\u3057\u3066\u3044\u308b\u3088\u3046\u306a\u51fa\u529b\u306b\u306a\u3063\u3066\u3044\u308b\uff08\u300c\u5b9a\u7fa9\u300d\u306e\u90e8\u5206\uff09\u3002\u4e00\u65b9RAG\u3042\u308a\u306e\u5834\u5408\u306f\u3001\u30c9\u30ad\u30e5\u30e1\u30f3\u30c8\u306b\u3042\u308b\u77e5\u898b\u3092\u57fa\u306b\uff12\u3064\u306ecost\u306e\u9055\u3044\u3092\u8ff0\u3079\u3066\u3044\u308b\u305f\u3081\u3001\u56de\u7b54\u304c\u81ea\u7136\u306a\u4f1a\u8a71\u306e\u3088\u3046\u306b\u611f\u3058\u305f\u3002<\/li>\r\n<\/ul>\r\n<p>\u3053\u306e\u3088\u3046\u306a\u80cc\u666f\u304b\u3089\u3001\u65e2\u306bLLM\u5185\u306b\u3042\u308b\u3088\u3046\u306a\u77e5\u8b58\u3067\u3042\u3063\u3066\u3082\u3001RAG\u3092\u5b9f\u88c5\u3059\u308b\u3053\u3068\u3067\u3001\u3088\u308a\u81ea\u7136\u3067\u30e6\u30fc\u30b6\u30fc\u304c\u7406\u89e3\u3057\u3084\u3059\u3044\u56de\u7b54\u3092\u751f\u6210\u3059\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u306e\u3067\u306f\u306a\u3044\u304b\u3068\u8003\u3048\u3066\u3044\u307e\u3059\u3002<\/p>\r\n<p><!-- notionvc: 580813d4-72d8-47e1-9cca-e074a18dfad9 --><\/p>","protected":false},"excerpt":{"rendered":"<p>\u672c\u8a18\u4e8b\u3067\u306f\u3001\u5b9f\u969b\u306b\u5916\u90e8\u30c7\u30fc\u30bf\u3092\u7528\u610f\u3057\u3001RAG\u306e\u5b9f\u88c5\u30d5\u30ed\u30fc\u3092\u8a73\u3057\u304f\u8aac\u660e\u3057\u3001GCP\u74b0\u5883\u3092\u4f7f\u7528\u3057\u305f\u958b\u767a\u74b0\u5883\u306e\u69cb\u7bc9\u304b\u3089\u3001\u30b7\u30b9\u30c6\u30e0\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u3001\u30c6\u30ad\u30b9\u30c8\u306e\u62bd\u51fa\u3001\u30a8\u30f3\u30d9\u30c7\u30a3\u30f3\u30b0\u51e6\u7406\u3001\u30d9\u30af\u30c8\u30eb\u691c\u7d22\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u4f5c\u6210\u307e\u3067\u306e\u30d7\u30ed\u30bb\u30b9\u3092\u89e3 [&hellip;]<\/p>\n","protected":false},"author":85,"featured_media":3964,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","swell_btn_cv_data":"","footnotes":"","_wp_rev_ctl_limit":""},"categories":[1249],"tags":[376,207,564,380,563,469],"class_list":["post-4548","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-knowledge","tag-chatgpt","tag-google-cloud","tag-llm","tag-openai","tag-rag","tag-vertexai"],"_links":{"self":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/4548","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/users\/85"}],"replies":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/comments?post=4548"}],"version-history":[{"count":0,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/4548\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media\/3964"}],"wp:attachment":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media?parent=4548"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/categories?post=4548"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/tags?post=4548"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}