{"id":9649,"date":"2026-09-15T16:42:24","date_gmt":"2026-09-15T07:42:24","guid":{"rendered":"https:\/\/since2020.jp\/media\/?p=9649"},"modified":"2026-09-15T17:36:43","modified_gmt":"2026-09-15T08:36:43","slug":"generative_model_rnn","status":"publish","type":"post","link":"https:\/\/since2020.jp\/media\/generative_model_rnn\/","title":{"rendered":"\u3010\u6df1\u5c64\u5b66\u7fd2\u3011\u751f\u6210\u30e2\u30c7\u30eb\u3092\u7406\u89e3\u3059\u308b_RNN"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">\u306f\u3058\u3081\u306b<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">\u751f\u6210\u30e2\u30c7\u30eb\u3068\u306f<\/h3>\n\n\n\n<p>\u751f\u6210\u30e2\u30c7\u30eb\u3068\u306f\u3001\u30c7\u30fc\u30bf\u306e\u5206\u5e03\u3092\u5b66\u7fd2\u3057\u3001\u65b0\u3057\u3044\u30c7\u30fc\u30bf\u3092\u751f\u6210\u3059\u308b\u30e2\u30c7\u30eb\u3067\u3059\u3002\u524d\u56de\u306e\u8a18\u4e8b\u3067\u306f\u305d\u306e\u6700\u3082\u30b7\u30f3\u30d7\u30eb\u306a\u5b9f\u88c5\u3068\u3057\u3066<strong>\u30de\u30eb\u30b3\u30d5\u9023\u9396<\/strong>\u3092\u53d6\u308a\u4e0a\u3052\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u30de\u30eb\u30b3\u30d5\u9023\u9396\u3068\u306e\u95a2\u4fc2<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">\u7c21\u5358\u306a\u5fa9\u7fd2<\/h4>\n\n\n\n<p>\u30de\u30eb\u30b3\u30d5\u9023\u9396\u306f\u300c\u76f4\u524d\u306eN\u6587\u5b57\u304b\u3089\u6b21\u306e\u6587\u5b57\u306e\u78ba\u7387\u3092\u6c42\u3081\u3001\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u3066\u6587\u7ae0\u3092\u751f\u6210\u3059\u308b\u300d\u30e2\u30c7\u30eb\u3067\u3057\u305f\u3002\u5916\u90e8\u30e9\u30a4\u30d6\u30e9\u30ea\u4e0d\u8981\u3067\u5b9f\u88c5\u3067\u304d\u3001\u751f\u6210\u30e2\u30c7\u30eb\u306e\u672c\u8cea\u3067\u3042\u308b\u300c\u78ba\u7387\u5206\u5e03\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u300d\u3092\u4f53\u611f\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">\u30de\u30eb\u30b3\u30d5\u9023\u9396\u306e\u8ab2\u984c\u611f<\/h4>\n\n\n\n<p>\u30de\u30eb\u30b3\u30d5\u9023\u9396\u306b\u306f\u6839\u672c\u7684\u306a\u9650\u754c\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>\u6587\u8108\u304c\u77ed\u3044<\/strong>: \u76f4\u524d\u306eN\u6587\u5b57\u3057\u304b\u53c2\u7167\u3067\u304d\u306a\u3044<\/li>\n\n\n\n<li><strong>\u30b3\u30fc\u30d1\u30b9\u5916\u3092\u751f\u6210\u3067\u304d\u306a\u3044<\/strong>: \u5b66\u7fd2\u30c7\u30fc\u30bf\u306b\u306a\u3044\u7d44\u307f\u5408\u308f\u305b\u306f\u51fa\u529b\u4e0d\u53ef\u80fd<\/li>\n\n\n\n<li><strong>\u610f\u5473\u3092\u7406\u89e3\u3057\u306a\u3044<\/strong>: \u78ba\u7387\u306e\u9ad8\u4f4e\u3092\u899a\u3048\u3066\u3044\u308b\u3060\u3051<\/li>\n<\/ul>\n\n\n\n<p>RNN\u306f\u3053\u306e\u300c\u6587\u8108\u304c\u77ed\u3044\u300d\u554f\u984c\u3092\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3067\u89e3\u6c7a\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">\u554f\u984c\u8a2d\u5b9a<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">\u554f\u984c\u8a2d\u5b9a<\/h3>\n\n\n\n<p>\u524d\u56de\u3068\u540c\u69d8\u306b\u300c\u65e5\u672c\u8a9e\u30c6\u30ad\u30b9\u30c8\u306e\u751f\u6210\u300d\u3092\u554f\u984c\u3068\u3057\u3066\u6271\u3044\u307e\u3059\u3002\u30b3\u30fc\u30d1\u30b9\u306f\u590f\u76ee\u6f31\u77f3\u300c\u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u300d\u306e\u5192\u982d10\u6587\uff08253\u6587\u5b57\uff09\u3067\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u4f55\u3092\u76ee\u7684\u306b\u3057\u3066\u3044\u308b\u304b<\/h3>\n\n\n\n<p>RNN\u304c\u6301\u3064**\u96a0\u308c\u72b6\u614b\uff08hidden state\uff09**\u3068\u3044\u3046\u300c\u8a18\u61b6\u300d\u306e\u4ed5\u7d44\u307f\u3092\u7406\u89e3\u3057\u3001\u30de\u30eb\u30b3\u30d5\u9023\u9396\u3068\u306e\u9055\u3044\u3092\u5b9f\u88c5\u30ec\u30d9\u30eb\u3067\u78ba\u8a8d\u3059\u308b\u3053\u3068\u304c\u76ee\u7684\u3067\u3059\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">RNN\u3068\u306f<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">RNN\u306e\u6982\u8981<\/h3>\n\n\n\n<p>RNN\uff08Recurrent Neural Network\uff1a\u30ea\u30ab\u30ec\u30f3\u30c8\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\uff09\u306f\u3001\u7cfb\u5217\u30c7\u30fc\u30bf\u3092\u51e6\u7406\u3059\u308b\u305f\u3081\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3067\u3059\u3002\u6700\u5927\u306e\u7279\u5fb4\u306f<strong>\u96a0\u308c\u72b6\u614b<\/strong>\u3068\u3044\u3046\u8a18\u61b6\u3092\u6301\u3061\u3001\u904e\u53bb\u306e\u60c5\u5831\u3092\u5f15\u304d\u7d99\u304e\u306a\u304c\u3089\u51e6\u7406\u3067\u304d\u308b\u70b9\u3067\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u30de\u30eb\u30b3\u30d5\u9023\u9396:  &#091;\u543e]&#091;\u8f29] \u2192 \u6b21\u306e\u6587\u5b57   \uff08\u76f4\u524d2\u6587\u5b57\u306e\u307f\u53c2\u7167\uff09\n\nRNN:  &#091;\u543e]&#091;\u8f29]&#091;\u306f]&#091;\u732b]&#091;\u3067]&#091;\u3042]&#091;\u308b] \u2192 \u6b21\u306e\u6587\u5b57\n        \u2191 \u904e\u53bb\u306e\u5168\u6587\u8108\u3092\u96a0\u308c\u72b6\u614b\u3068\u3057\u3066\u84c4\u7a4d \u2191\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">RNN\u306e\u4ed5\u7d44\u307f<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">\u9806\u4f1d\u64ad\uff08Forward Pass\uff09<\/h4>\n\n\n\n<p>RNN\u306f\u5404\u30bf\u30a4\u30e0\u30b9\u30c6\u30c3\u30d7\u3067\u4ee5\u4e0b\u306e\u8a08\u7b97\u3092\u884c\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>h_t = tanh(W_h \u00d7 h_{t-1} + W_x \u00d7 x_t + b)\ny_t = W_y \u00d7 h_t\n\nh_t    : \u73fe\u5728\u306e\u96a0\u308c\u72b6\u614b\uff08\u8a18\u61b6\uff09\nh_{t-1}: \u524d\u306e\u30bf\u30a4\u30e0\u30b9\u30c6\u30c3\u30d7\u306e\u96a0\u308c\u72b6\u614b\uff08\u904e\u53bb\u306e\u8a18\u61b6\uff09\nx_t    : \u73fe\u5728\u306e\u5165\u529b\u6587\u5b57\uff08\u57cb\u3081\u8fbc\u307f\u30d9\u30af\u30c8\u30eb\uff09\ny_t    : \u6b21\u306e\u6587\u5b57\u306e\u4e88\u6e2c\u30b9\u30b3\u30a2\uff08\u30ed\u30b8\u30c3\u30c8\uff09\n<\/code><\/pre>\n\n\n\n<p>\u5165\u529b\u6587\u5b57\u3092\u53d7\u3051\u53d6\u308b\u305f\u3073\u306b\u96a0\u308c\u72b6\u614b\u304c\u66f4\u65b0\u3055\u308c\u3001\u904e\u53bb\u306e\u6587\u8108\u304c\u84c4\u7a4d\u3055\u308c\u3066\u3044\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">\u9006\u4f1d\u64ad\uff08Backpropagation Through Time\uff09<\/h4>\n\n\n\n<p>RNN\u306e\u9006\u4f1d\u64ad\uff08BPTT\uff09\u3092\u7406\u89e3\u3059\u308b\u305f\u3081\u306b\u306f\u3001\u307e\u305a\u300c$W_h$ \u304c\u5168\u30b9\u30c6\u30c3\u30d7\u3067\u5171\u6709\u3055\u308c\u3066\u3044\u308b\u300d\u3053\u3068\u304c\u91cd\u8981\u3067\u3059\u3002<\/p>\n\n\n\n<p>$$<br>h_1 = \\tanh(W_h h_0 + W_x x_1)<br>$$<\/p>\n\n\n\n<p>$$<br>h_2 = \\tanh(W_h h_1 + W_x x_2)<br>$$<\/p>\n\n\n\n<p>$$<br>h_3 = \\tanh(W_h h_2 + W_x x_3)<br>$$<\/p>\n\n\n\n<p>\u3044\u305a\u308c\u3082\u540c\u3058 $W_h$ \u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059\u3002$W_h$ \u306f\u5168\u30b9\u30c6\u30c3\u30d7\u3067\u4f7f\u3044\u56de\u3055\u308c\u3066\u3044\u308b\u305f\u3081\u3001\u300c$W_h$ \u3092\u5c11\u3057\u5909\u3048\u305f\u3068\u304d\u3001\u640d\u5931\u306b\u3069\u3046\u5f71\u97ff\u3059\u308b\u304b\u300d\u3092\u6c42\u3081\u308b\u306b\u306f<strong>\u5168\u30b9\u30c6\u30c3\u30d7\u3078\u306e\u5f71\u97ff\u3092\u8db3\u3057\u5408\u308f\u305b\u308b<\/strong>\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002\u4ee5\u4e0b\u3067\u306f\u3001\u306a\u305c\u8db3\u3057\u7b97\u306b\u306a\u308b\u306e\u304b\u3092\u7701\u7565\u305b\u305a\u306b\u8ffd\u3063\u3066\u3044\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>0. \u8a18\u53f7\u306e\u6574\u7406<\/strong><\/p>\n\n\n\n<p>\u5404\u30b9\u30c6\u30c3\u30d7\u306e\u8a08\u7b97\u3092\u6b21\u306e\u3088\u3046\u306b\u5206\u89e3\u3057\u3066\u304a\u304d\u307e\u3059\u3002$z_t$ \u306f\u6d3b\u6027\u5316\u95a2\u6570\u306b\u5165\u308b\u524d\u306e\u5024\u3067\u3059\u3002<\/p>\n\n\n\n<p>$$<br>z_t = W_h h_{t-1} + W_x x_t + b<br>$$<\/p>\n\n\n\n<p>$$<br>h_t = \\tanh(z_t)<br>$$<\/p>\n\n\n\n<p>$$<br>L = L_1 + L_2 + L_3<br>$$<\/p>\n\n\n\n<p>\uff08\u5b9f\u969b\u306e\u5b9f\u88c5\u3067\u306f20\u30b9\u30c6\u30c3\u30d7\u3067\u3059\u304c\u3001\u3053\u3053\u3067\u306f\u4ed5\u7d44\u307f\u304c\u898b\u3048\u3084\u3059\u3044\u3088\u30463\u30b9\u30c6\u30c3\u30d7\u3067\u8003\u3048\u307e\u3059\u3002\u307e\u305f $h_0$ \u306f\u521d\u671f\u5024\u3067\u3042\u308a\u3001$W_h$ \u306b\u4f9d\u5b58\u3057\u306a\u3044\u5b9a\u6570\u3068\u3057\u3066\u6271\u3044\u307e\u3059\u3002\uff09<\/p>\n\n\n\n<p>\u6c42\u3081\u305f\u3044\u306e\u306f $\\dfrac{\\partial L}{\\partial W_h}$ \u3067\u3059\u3002$L$ \u304c\u548c\u306e\u5f62\u3092\u3057\u3066\u3044\u308b\u306e\u3067\u3001\u5fae\u5206\u306e\u7dda\u5f62\u6027\u3088\u308a\u307e\u305a\u6b21\u306e\u3088\u3046\u306b\u5206\u89e3\u3067\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L}{\\partial W_h} = \\frac{\\partial L_1}{\\partial W_h} + \\frac{\\partial L_2}{\\partial W_h} + \\frac{\\partial L_3}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>\u3053\u306e\u53f3\u8fba\u306e3\u9805\u3092\u3001\u305d\u308c\u305e\u308c\u8a08\u7b97\u3057\u3066\u3044\u304d\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>1. \u30b9\u30c6\u30c3\u30d71\uff08t=1\uff09: \u7d4c\u8def\u306f1\u672c\u3060\u3051<\/strong><\/p>\n\n\n\n<p>$L_1$ \u306e\u4f9d\u5b58\u95a2\u4fc2\u306f $L_1 \\leftarrow h_1 \\leftarrow z_1$ \u306a\u306e\u3067\u3001\u9023\u9396\u5f8b\u3088\u308a\u6b21\u306e\u5f62\u304c\u6210\u308a\u7acb\u3061\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_1}{\\partial W_h} = \\frac{\\partial L_1}{\\partial h_1}\\cdot\\frac{\\partial h_1}{\\partial z_1}\\cdot\\frac{\\partial z_1}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>$z_1 = W_h h_0 + W_x x_1 + b$ \u306b\u304a\u3044\u3066\u3001$h_0$ \u306f\u5b9a\u6570\u306a\u306e\u3067 $W_h$ \u304c\u767b\u5834\u3059\u308b\u306e\u306f1\u7b87\u6240\u3060\u3051\u3067\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial z_1}{\\partial W_h} = h_0^{\\top}<br>$$<\/p>\n\n\n\n<p>\u3088\u3063\u3066\uff1a<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_1}{\\partial W_h} = \\left(\\frac{\\partial L_1}{\\partial h_1}\\odot\\frac{\\partial h_1}{\\partial z_1}\\right)h_0^{\\top}<br>$$<\/p>\n\n\n\n<p>\u3053\u3053\u3067\u3001\u53f3\u8fba\u306e\u3046\u3061\u5148\u982d\u306e $\\dfrac{\\partial L_1}{\\partial h_1}$ \u3092\u9664\u3044\u305f\u90e8\u5206\u306f\u3001\u5b9f\u306f\u305d\u306e\u307e\u307e $\\dfrac{\\partial h_1}{\\partial W_h}$\uff08$h_1$ \u3092 $W_h$ \u3067\u76f4\u63a5\u5fae\u5206\u3057\u305f\u3082\u306e\uff09\u306b\u3082\u306a\u3063\u3066\u3044\u307e\u3059\u3002\u3053\u306e\u526f\u7523\u7269\u3092\u6b21\u306e\u30b9\u30c6\u30c3\u30d7\u3067\u4f7f\u3044\u56de\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial h_1}{\\partial W_h} = \\frac{\\partial h_1}{\\partial z_1}\\odot h_0^{\\top}<br>$$<\/p>\n\n\n\n<p><strong>2. \u30b9\u30c6\u30c3\u30d72\uff08t=2\uff09: \u7d4c\u8def\u304c2\u672c\u306b\u5897\u3048\u308b<\/strong><\/p>\n\n\n\n<p>$L_2$ \u3082\u51fa\u767a\u70b9\u306f\u540c\u3058\u5f62\u3067\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_2}{\\partial W_h} = \\frac{\\partial L_2}{\\partial h_2}\\cdot\\frac{\\partial h_2}{\\partial z_2}\\cdot\\frac{\\partial z_2}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>\u9055\u3044\u304c\u51fa\u308b\u306e\u306f\u6700\u5f8c\u306e\u9805\u3067\u3059\u3002$z_2 = W_h h_1 + W_x x_2 + b$ \u3067\u3059\u304c\u3001\u4eca\u56de\u306f $h_1$ \u304c\u5b9a\u6570\u3067\u306f\u306a\u304f $W_h$ \u306e\u95a2\u6570\uff08$h_1=\\tanh(W_h h_0 + W_x x_1+b)$\uff09\u3067\u3059\u3002\u305d\u306e\u305f\u3081\u300c$W_h$ \u81ea\u8eab\u300d\u3068\u300c$h_1$ \u306e\u4e2d\u306e $W_h$\u300d\u3068\u3044\u30462\u7b87\u6240\u304c\u767b\u5834\u3057\u3001\u6b21\u306e\u3088\u3046\u306b\u5c55\u958b\u3055\u308c\u307e\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"748\" height=\"147\" src=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2026\/09\/image.png\" alt=\"\" class=\"wp-image-9652\" srcset=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2026\/09\/image.png 748w, https:\/\/since2020.jp\/media\/wp-content\/uploads\/2026\/09\/image-300x59.png 300w\" sizes=\"(max-width: 748px) 100vw, 748px\" \/><\/figure>\n\n\n\n<p>\u2460\u306f\u300c$W_h h_1$ \u306e\u639b\u3051\u3066\u3044\u308b\u65b9\u306e $W_h$\u300d\u3092 $h_1$ \u3092\u5b9a\u6570\u3068\u307f\u306a\u3057\u3066\u5fae\u5206\u3057\u305f\u9805\u3001\u2461\u306f $h_1$ \u306e\u4e2d\u8eab\u306b\u3082 $W_h$ \u304c\u57cb\u3081\u8fbc\u307e\u308c\u3066\u3044\u308b\u3053\u3068\u306b\u3088\u308b\u9805\u3067\u3059\u3002\u2461\u306e $\\dfrac{\\partial h_1}{\\partial W_h}$ \u306f\u3001\u30b9\u30c6\u30c3\u30d71\u306e\u6700\u5f8c\u3067\u8a08\u7b97\u6e08\u307f\u306e\u3082\u306e\u3092\u305d\u306e\u307e\u307e\u4f7f\u3048\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial z_2}{\\partial W_h} = h_1^{\\top} + W_h\\left[\\frac{\\partial h_1}{\\partial z_1}\\odot h_0^{\\top}\\right]<br>$$<\/p>\n\n\n\n<p>\u307e\u3068\u3081\u308b\u3068\uff1a<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_2}{\\partial W_h} = \\left(\\frac{\\partial L_2}{\\partial h_2}\\odot\\frac{\\partial h_2}{\\partial z_2}\\right)\\left(h_1^{\\top} + W_h\\left[\\frac{\\partial h_1}{\\partial z_1}\\odot h_0^{\\top}\\right]\\right)<br>$$<\/p>\n\n\n\n<p>\u3053\u306e\u5f0f\u3092\u5c55\u958b\u3059\u308b\u30682\u3064\u306e\u9805\u304c\u51fa\u3066\u304d\u307e\u3059\u3002\u3053\u308c\u304c\u300c\u7d4c\u8def\u304c2\u672c\u306b\u306a\u308b\u300d\u3053\u3068\u306e\u6b63\u4f53\u3067\u3059\u3002\u540c\u69d8\u306b\u3001\u3053\u3053\u3067\u3082\u526f\u7523\u7269\u3068\u3057\u3066 $\\dfrac{\\partial h_2}{\\partial W_h}$ \u304c\u8a08\u7b97\u3067\u304d\u3001\u6b21\u306e\u30b9\u30c6\u30c3\u30d7\u3067\u518d\u5229\u7528\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>3. \u30b9\u30c6\u30c3\u30d73\uff08t=3\uff09: \u7d4c\u8def\u304c3\u672c\u306b\u306a\u308b<\/strong><\/p>\n\n\n\n<p>$L_3$ \u3082\u51fa\u767a\u70b9\u306f\u540c\u3058\u5f62\u3067\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_3}{\\partial W_h} = \\frac{\\partial L_3}{\\partial h_3}\\cdot\\frac{\\partial h_3}{\\partial z_3}\\cdot\\frac{\\partial z_3}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>\u6700\u5f8c\u306e\u9805\u3092\u8a08\u7b97\u3057\u307e\u3059\u3002$z_3 = W_h h_2 + W_x x_3 + b$ \u3067\u3001$h_2$ \u3082\u307e\u305f $W_h$ \u306e\u95a2\u6570\uff08$h_2=\\tanh(W_h h_1 + W_x x_2+b)$\uff09\u306a\u306e\u3067\u3001\u30b9\u30c6\u30c3\u30d72\u3068\u540c\u3058\u7406\u5c48\u3067\u300c$W_h$ \u81ea\u8eab\u300d\u3068\u300c$h_2$ \u306e\u4e2d\u306e $W_h$\u300d\u306e2\u7b87\u6240\u304c\u73fe\u308c\u3001\u6b21\u306e\u3088\u3046\u306b\u5c55\u958b\u3055\u308c\u307e\u3059\u3002<\/p>\n\n\n\n\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"769\" height=\"120\" src=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2026\/09\/image-1.png\" alt=\"\" class=\"wp-image-9653\" srcset=\"https:\/\/since2020.jp\/media\/wp-content\/uploads\/2026\/09\/image-1.png 769w, https:\/\/since2020.jp\/media\/wp-content\/uploads\/2026\/09\/image-1-300x47.png 300w\" sizes=\"(max-width: 769px) 100vw, 769px\" \/><\/figure>\n\n\n\n<p>\u2461\u306e $\\dfrac{\\partial h_2}{\\partial W_h}$ \u306f\u3001\u30b9\u30c6\u30c3\u30d72\u306e\u6700\u5f8c\u3067\u8a08\u7b97\u6e08\u307f\u306e\u526f\u7523\u7269\u3092\u305d\u306e\u307e\u307e\u4f7f\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial h_2}{\\partial W_h} = \\frac{\\partial h_2}{\\partial z_2}\\odot\\left(h_1^{\\top} + W_h\\left[\\frac{\\partial h_1}{\\partial z_1}\\odot h_0^{\\top}\\right]\\right)<br>$$<\/p>\n\n\n\n<p>\u3053\u308c\u3092\u4ee3\u5165\u3059\u308b\u3068\uff1a<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial z_3}{\\partial W_h} = h_2^{\\top} + W_h\\left(\\frac{\\partial h_2}{\\partial z_2}\\odot\\left(h_1^{\\top} + W_h\\left[\\frac{\\partial h_1}{\\partial z_1}\\odot h_0^{\\top}\\right]\\right)\\right)<br>$$<\/p>\n\n\n\n<p>\u3088\u3063\u3066\uff1a<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_3}{\\partial W_h} = \\left(\\frac{\\partial L_3}{\\partial h_3}\\odot\\frac{\\partial h_3}{\\partial z_3}\\right)\\left(h_2^{\\top} + W_h\\left(\\frac{\\partial h_2}{\\partial z_2}\\odot\\left(h_1^{\\top} + W_h\\left[\\frac{\\partial h_1}{\\partial z_1}\\odot h_0^{\\top}\\right]\\right)\\right)\\right)<br>$$<\/p>\n\n\n\n<p>\u3053\u308c\u3092\u5c55\u958b\u3059\u308b\u3068\u3001$\\dfrac{\\partial L_3}{\\partial W_h}$ \u306e\u4e2d\u306b\u306f3\u3064\u306e\u9805\u304c\u5165\u308c\u5b50\u3067\u51fa\u3066\u304d\u307e\u3059\u3002$t$ \u304c\u5897\u3048\u308b\u305f\u3073\u306b\u3001$W_h$ \u304c\u57cb\u3081\u8fbc\u307e\u308c\u3066\u3044\u308b\u7b87\u6240\u306e\u6570\u3060\u3051\u9805\u304c1\u3064\u305a\u3064\u5897\u3048\u3066\u3044\u304f\u306e\u304c\u5206\u304b\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>4. \u30d1\u30bf\u30fc\u30f3\u306e\u4e00\u822c\u5316\uff1a\u306a\u305c\u8db3\u3057\u7b97\u306a\u306e\u304b<\/strong><\/p>\n\n\n\n<p>\u3053\u3053\u307e\u3067\u306e\u8a08\u7b97\u306f\u3059\u3079\u3066\u3001\u6b21\u306e\u5171\u901a\u306e\u5f62\u304b\u3089\u59cb\u307e\u3063\u3066\u3044\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_t}{\\partial W_h} = \\frac{\\partial L_t}{\\partial h_t}\\cdot\\frac{\\partial h_t}{\\partial z_t}\\cdot\\frac{\\partial z_t}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>\u9055\u3044\u306f\u6700\u5f8c\u306e\u9805 $\\dfrac{\\partial z_t}{\\partial W_h}$ \u3092\u3069\u3053\u307e\u3067\u518d\u5e30\u7684\u306b\u5c55\u958b\u3059\u308b\u304b\u3060\u3051\u3067\u3059\u3002\u3053\u308c\u306f\u300c\u540c\u3058\u5909\u6570 $W_h$ \u304c\u8907\u6570\u306e\u7b87\u6240\uff08$t=1,\\dots,t$\uff09\u306b\u57cb\u3081\u8fbc\u307e\u308c\u305f\u5408\u6210\u95a2\u6570\u3092\u5fae\u5206\u3059\u308b\u3068\u3001\u57cb\u3081\u8fbc\u307e\u308c\u305f\u7b87\u6240\u306e\u6570\u3060\u3051\u548c\u304c\u751f\u307e\u308c\u308b\u300d\u3068\u3044\u3046\u591a\u5909\u6570\u306e\u9023\u9396\u5f8b\uff08\u5168\u5fae\u5206\uff09\u306e\u6027\u8cea\u305d\u306e\u3082\u306e\u3067\u3059\u3002\u6574\u7406\u3059\u308b\u3068\u3001\u3059\u3079\u3066\u306e\u9805\u306f\u6b21\u306e\u5f62\u306b\u307e\u3068\u3081\u3089\u308c\u307e\u3059\uff08$k$ \u306f\u300c$W_h$ \u304c\u76f4\u63a5\u4f7f\u308f\u308c\u3066\u3044\u308b\u6642\u523b\u300d\uff09\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_t}{\\partial W_h} = \\sum_{k=1}^{t}\\frac{\\partial L_t}{\\partial h_t}\\cdot\\frac{\\partial h_t}{\\partial h_k}\\cdot\\left(\\frac{\\partial h_k}{\\partial z_k}\\odot h_{k-1}^{\\top}\\right)<br>$$<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial h_t}{\\partial h_k} = \\prod_{i=k+1}^{t}\\left[\\frac{\\partial h_i}{\\partial z_i}\\odot W_h\\right], \\qquad \\frac{\\partial h_t}{\\partial h_t}=1<br>$$<\/p>\n\n\n\n<p><strong>5. \u5168\u30b9\u30c6\u30c3\u30d7\u3092\u8db3\u3057\u5408\u308f\u305b\u3066\u6700\u7d42\u5f62\u306b\u307e\u3068\u3081\u308b<\/strong><\/p>\n\n\n\n<p>\u6700\u521d\u306e\u5206\u89e3 $\\frac{\\partial L}{\\partial W_h}=\\sum_t \\frac{\\partial L_t}{\\partial W_h}$ \u306b\u4e0a\u306e\u5f0f\u3092\u4ee3\u5165\u3059\u308b\u3068\u3001$t$ \u3068 $k$ \u306b\u3064\u3044\u3066\u306e\u4e8c\u91cd\u306e\u548c\uff08$1\\le k\\le t\\le T$\uff09\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L}{\\partial W_h} = \\sum_{t=1}^{T}\\sum_{k=1}^{t}\\frac{\\partial L_t}{\\partial h_t}\\cdot\\frac{\\partial h_t}{\\partial h_k}\\cdot\\left(\\frac{\\partial h_k}{\\partial z_k}\\odot h_{k-1}^{\\top}\\right)<br>$$<\/p>\n\n\n\n<p>$k$ \u3092\u56fa\u5b9a\u3057\u3066 $t=k,\\dots,T$ \u3092\u5148\u306b\u8db3\u3059\u3068\u3001\u305d\u306e\u4e2d\u62ec\u5f27\u306e\u4e2d\u8eab\u304c\u3061\u3087\u3046\u3069\u300cBPTT\u3067\u9006\u9806\u306b\u8a08\u7b97\u3059\u308b $\\dfrac{\\partial L}{\\partial h_k}$\u300d\u306e\u5b9a\u7fa9\u305d\u306e\u3082\u306e\u306b\u306a\u308a\u307e\u3059\uff08$h_k$ \u306f $k$ \u756a\u76ee\u4ee5\u964d\u306e\u3059\u3079\u3066\u306e\u640d\u5931\u306b\u5f71\u97ff\u3059\u308b\u305f\u3081\uff09\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L}{\\partial W_h} = \\sum_{k=1}^{T}\\left[\\sum_{t=k}^{T}\\frac{\\partial L_t}{\\partial h_t}\\cdot\\frac{\\partial h_t}{\\partial h_k}\\right]\\cdot\\left(\\frac{\\partial h_k}{\\partial z_k}\\odot h_{k-1}^{\\top}\\right) = \\sum_{t=1}^{T}\\left(\\frac{\\partial L}{\\partial h_t}\\odot\\frac{\\partial h_t}{\\partial z_t}\\right)h_{t-1}^{\\top}<br>$$<\/p>\n\n\n\n<p>\u3053\u308c\u304c\u6700\u7d42\u7684\u306a\u52fe\u914d\u306e\u5f0f\u3067\u3059\u3002$L=\\sum_t L_t$ \u3068\u3044\u3046<strong>\u640d\u5931\u306e\u5b9a\u7fa9\u306b\u3088\u308b\u548c<\/strong>\u3068\u3001$W_h$ \u304c\u5168\u6642\u523b\u306b\u57cb\u3081\u8fbc\u307e\u308c\u3066\u3044\u308b\u3053\u3068\u306b\u3088\u308b<strong>\u9023\u9396\u5f8b\u306e\u548c<\/strong>\u3001\u3053\u306e2\u3064\u306e\u548c\u304c\u6574\u7406\u3055\u308c\u3066\u3001\u898b\u305f\u76ee\u306b\u306f\u6642\u523b\u306b\u3064\u3044\u3066\u306e1\u3064\u306e\u548c\u306b\u307e\u3068\u307e\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>\u5168\u30b9\u30c6\u30c3\u30d7\u306e\u640d\u5931\u3092\u4f7f\u3063\u305f\u6700\u7d42\u7684\u306a\u66f4\u65b0<\/strong><\/p>\n\n\n\n<p>\u5404\u30b9\u30c6\u30c3\u30d7\u306e\u640d\u5931 $L_1$\u301c$L_{20}$ \u305d\u308c\u305e\u308c\u304b\u3089\u540c\u69d8\u306b $W_h$ \u3078\u306e\u52fe\u914d\u3092\u8a08\u7b97\u3057\u3001\u5168\u3066\u8db3\u3057\u5408\u308f\u305b\u305f\u3082\u306e\u304c\u6700\u7d42\u7684\u306a\u52fe\u914d\u3067\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L}{\\partial W_h} = \\frac{\\partial L_1}{\\partial W_h} + \\frac{\\partial L_2}{\\partial W_h} + \\cdots + \\frac{\\partial L_{20}}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>$$<br>W_h \\leftarrow W_h &#8211; \\eta\\,\\frac{\\partial L}{\\partial W_h}<br>$$<\/p>\n\n\n\n<p>\uff08$\\eta$ \u306f\u5b66\u7fd2\u7387\uff09\u3053\u308c\u30921\u30a8\u30dd\u30c3\u30af\u3054\u3068\u306b1\u56de\u884c\u3044\u307e\u3059\u3002PyTorch \u3067\u306f <code>loss.backward()<\/code> \u304c\u5168\u7d4c\u8def\u306e\u52fe\u914d\u3092\u81ea\u52d5\u8a08\u7b97\u3057\u3001<code>optimizer.step()<\/code> \u304c\u4e0a\u8a18\u306e\u66f4\u65b0\u5f0f\u3092\u5b9f\u884c\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>\u52fe\u914d\u6d88\u5931\u554f\u984c<\/strong><\/p>\n\n\n\n<p>\u3053\u3053\u3067\u6ce8\u610f\u3057\u305f\u3044\u306e\u306f\u3001\u300c\u60c5\u5831\u304c\u6d88\u3048\u308b\u300d\u3053\u3068\u3068\u300c\u52fe\u914d\uff08\u5b66\u7fd2\u4fe1\u53f7\uff09\u304c\u6d88\u3048\u308b\u300d\u3053\u3068\u306f\u5225\u7269\u3060\u3068\u3044\u3046\u70b9\u3067\u3059\u3002<\/p>\n\n\n\n<p>\u30d5\u30a9\u30ef\u30fc\u30c9\uff08\u60c5\u5831\u306e\u4f1d\u308f\u308a\u65b9\uff09\u3067\u306f\u3001$h_1 \\to h_2 \\to \\cdots \\to h_{20}$ \u3068\u3044\u3046\u9023\u9396\u304c\u3042\u308b\u305f\u3081\u30001\u6587\u5b57\u76ee\u306e\u60c5\u5831\u306f\uff08$\\tanh$ \u3067\u5727\u7e2e\u3055\u308c\u306a\u304c\u3089\u3082\uff09$h_{20}$ \u307e\u3067\u304d\u3061\u3093\u3068\u4f1d\u308f\u3063\u3066\u304a\u308a\u3001\u60c5\u5831\u81ea\u4f53\u304c\u6d88\u3048\u3066\u3044\u308b\u308f\u3051\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\u554f\u984c\u306f\u30d0\u30c3\u30af\u30ef\u30fc\u30c9\uff08\u52fe\u914d\u306e\u4f1d\u308f\u308a\u65b9\uff09\u306e\u65b9\u3067\u3059\u3002$k&lt;t$ \u306e\u3068\u304d $\\dfrac{\\partial L_t}{\\partial h_k}$ \u3092\u6c42\u3081\u308b\u306b\u306f\u3001$h_k$ \u304b\u3089 $h_t$ \u307e\u3067\u9023\u9396\u5f8b\u3067 $(t-k)$ \u56de\u3082\u30e4\u30b3\u30d3\u30a2\u30f3\u3092\u639b\u3051\u5408\u308f\u305b\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L_t}{\\partial h_k} = \\frac{\\partial L_t}{\\partial h_t}\\cdot\\prod_{i=k+1}^{t}\\frac{\\partial h_i}{\\partial h_{i-1}}, \\qquad \\frac{\\partial h_i}{\\partial h_{i-1}} = \\frac{\\partial h_i}{\\partial z_i}\\odot W_h<br>$$<\/p>\n\n\n\n<p>\u5404\u56e0\u5b50 $\\dfrac{\\partial h_i}{\\partial h_{i-1}}$ \u306e\u5927\u304d\u3055\u304c1\u672a\u6e80\uff08$\\tanh$ \u306e\u5fae\u5206\u306f\u6700\u5927\u3067\u30821\u672a\u6e80\uff09\u3060\u3068\u3059\u308b\u3068\u3001$(t-k)$ \u56de\u306e\u639b\u3051\u7b97\u3067\u5024\u306f\u6307\u6570\u7684\u306b\u6e1b\u8870\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<p>$$<br>\\left\\|\\frac{\\partial h_i}{\\partial h_{i-1}}\\right\\| \\approx 0.5 \\quad\\Longrightarrow\\quad \\left\\|\\prod_{i=k+1}^{t}\\frac{\\partial h_i}{\\partial h_{i-1}}\\right\\| \\approx 0.5^{\\,t-k}<br>$$<\/p>\n\n\n\n<p>$$<br>0.5^{1} = 0.50, \\qquad 0.5^{5} \\approx 0.03, \\qquad 0.5^{20} \\approx 0.001<br>$$<\/p>\n\n\n\n<p>\u3064\u307e\u308a\u3001$h_1$ \u306f $L_{20}$ \u306e\u8a08\u7b97\uff08\u30d5\u30a9\u30ef\u30fc\u30c9\uff09\u306b\u306f\u3061\u3083\u3093\u3068\u4f7f\u308f\u308c\u3066\u3044\u308b\u306e\u306b\u3001\u300c$h_1$ \u3092\u3069\u3046\u8abf\u6574\u3059\u308c\u3070 $L_{20}$ \u304c\u6539\u5584\u3059\u308b\u304b\u300d\u3092\u6559\u3048\u308b\u52fe\u914d $\\dfrac{\\partial L_{20}}{\\partial h_1}$ \u306f\u307b\u307c\u30bc\u30ed\u306b\u306a\u3063\u3066\u3057\u307e\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u524d\u7bc0\u3067\u5c0e\u3044\u305f\u6700\u7d42\u5f62\u3092\u601d\u3044\u51fa\u3059\u3068\uff1a<\/p>\n\n\n\n<p>$$<br>\\frac{\\partial L}{\\partial W_h} = \\sum_{k=1}^{T}\\left[\\sum_{t=k}^{T}\\frac{\\partial L_t}{\\partial h_t}\\cdot\\frac{\\partial h_t}{\\partial h_k}\\right]\\cdot\\left(\\frac{\\partial h_k}{\\partial z_k}\\odot h_{k-1}^{\\top}\\right)<br>$$<\/p>\n\n\n\n<p>\u5185\u5074\u306e\u548c $\\sum_{t=k}^{T}\\dfrac{\\partial L_t}{\\partial h_t}\\cdot\\dfrac{\\partial h_t}{\\partial h_k}$ \u3092\u898b\u308b\u3068\u3001$t-k$ \u304c\u5927\u304d\u3044\u9805\uff08$k$ \u304b\u3089\u9060\u3044\u672a\u6765\u306e\u640d\u5931\uff09\u307b\u3069 $\\dfrac{\\partial h_t}{\\partial h_k}$ \u306e\u6e1b\u8870\u306b\u3088\u308a\u307b\u3068\u3093\u3069\u5bc4\u4e0e\u3057\u306a\u304f\u306a\u308a\u307e\u3059\u3002\u8a00\u3044\u63db\u3048\u308b\u3068\u3001$k$ \u304c\u7cfb\u5217\u306e\u3069\u306e\u4f4d\u7f6e\u306b\u3042\u3063\u3066\u3082\u300c1\u301c\u6570\u30b9\u30c6\u30c3\u30d7\u5148\u306e\u640d\u5931\u300d\u304b\u3089\u306e\u52fe\u914d\u306f\u6e1b\u8870\u306a\u304f\u5c4a\u304d\u307e\u3059\u304c\u3001\u300c\u4f55\u5341\u30b9\u30c6\u30c3\u30d7\u3082\u5148\uff08\u307e\u305f\u306f\u524d\uff09\u306e\u640d\u5931\u300d\u3068\u306e\u95a2\u4fc2\u3092\u5b66\u7fd2\u3059\u308b\u4fe1\u53f7\u3060\u3051\u304c\u3001\u305d\u306e\u9694\u305f\u308a\u306e\u5927\u304d\u3055\u306b\u5fdc\u3058\u3066\u6307\u6570\u7684\u306b\u5f31\u304f\u306a\u308a\u307e\u3059\u3002\u3053\u308c\u304c<strong>\u52fe\u914d\u6d88\u5931\u554f\u984c<\/strong>\u3067\u3042\u308a\u3001RNN\u304c\u9577\u3044\u6587\u8108\uff08\u9060\u3044\u4f9d\u5b58\u95a2\u4fc2\uff09\u3092\u5b66\u7fd2\u3057\u306b\u304f\u3044\u6839\u672c\u7684\u306a\u539f\u56e0\u3067\u3059\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">\u5b9f\u9a13<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">\u5b9f\u9a13\u30c7\u30fc\u30bf<\/h3>\n\n\n\n<p>\u590f\u76ee\u6f31\u77f3\u300c\u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u300d\u306e\u5192\u982d10\u6587\u3092\u30b3\u30fc\u30d1\u30b9\u3068\u3057\u3066\u4f7f\u7528\u3057\u307e\u3059\u3002\u6587\u5b57\u7a2e\u306f74\u7a2e\u985e\u3067\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\"\"\"\n\u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u3002\u540d\u524d\u306f\u307e\u3060\u306a\u3044\u3002\n\u3069\u3053\u3067\u751f\u308c\u305f\u304b\u9813\u3068\u898b\u5f53\u304c\u3064\u304b\u306c\u3002\n\u4f55\u3067\u3082\u8584\u6697\u3044\u3058\u3081\u3058\u3081\u3057\u305f\u6240\u3067\u30cb\u30e3\u30fc\u30cb\u30e3\u30fc\u6ce3\u3044\u3066\u3044\u305f\u4e8b\u3060\u3051\u306f\u8a18\u61b6\u3057\u3066\u3044\u308b\u3002\n\u543e\u8f29\u306f\u3053\u3053\u3067\u59cb\u3081\u3066\u4eba\u9593\u3068\u3044\u3046\u3082\u306e\u3092\u898b\u305f\u3002\n\u3057\u304b\u3082\u3042\u3068\u3067\u805e\u304f\u3068\u305d\u308c\u306f\u66f8\u751f\u3068\u3044\u3046\u4eba\u9593\u4e2d\u3067\u4e00\u756a\u7370\u60aa\u306a\u7a2e\u65cf\u3067\u3042\u3063\u305f\u305d\u3046\u3060\u3002\n\u3053\u306e\u66f8\u751f\u3068\u3044\u3046\u306e\u306f\u6642\u3005\u6211\u3005\u3092\u6355\u3048\u3066\u716e\u3066\u98df\u3046\u3068\u3044\u3046\u8a71\u3067\u3042\u308b\u3002\n\u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u3002\u732b\u306e\u540d\u524d\u306f\u4f55\u3067\u3082\u3044\u3044\u3002\n\u732b\u306f\u3069\u3053\u3067\u3082\u5bdd\u308b\u3053\u3068\u304c\u3067\u304d\u308b\u3002\u732b\u306f\u81ea\u7531\u306a\u751f\u304d\u7269\u3060\u3002\n\u4eba\u9593\u306f\u732b\u3092\u898b\u3066\u7b11\u3046\u3002\u732b\u306f\u4eba\u9593\u3092\u898b\u3066\u8003\u3048\u308b\u3002\n\u543e\u8f29\u306f\u8003\u3048\u308b\u732b\u3067\u3042\u308b\u3002\u8003\u3048\u308b\u3053\u3068\u306f\u751f\u304d\u308b\u3053\u3068\u3060\u3002\n\"\"\"\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">\u30c8\u30fc\u30af\u30f3\u5316<\/h3>\n\n\n\n<p>\u30de\u30eb\u30b3\u30d5\u9023\u9396\u3068\u540c\u69d8\u306b\u6587\u5b57\u5358\u4f4d\u3067\u30c8\u30fc\u30af\u30f3\u5316\u3057\u307e\u3059\u3002\u305f\u3060\u3057RNN\u3067\u306f\u6587\u5b57\u3092\u305d\u306e\u307e\u307e\u6271\u308f\u305a\u3001<strong>\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306b\u5909\u63db<\/strong>\u3057\u3066\u304b\u3089**\u57cb\u3081\u8fbc\u307f\uff08Embedding\uff09**\u3067\u5bc6\u306a\u30d9\u30af\u30c8\u30eb\u306b\u5909\u63db\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>chars = sorted(set(corpus))        # \u8a9e\u5f59: 74\u6587\u5b57\nchar2idx = {ch: i for i, ch in enumerate(chars)}  # \u6587\u5b57\u2192\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\nidx2char = {i: ch for ch, i in char2idx.items()}  # \u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u2192\u6587\u5b57\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">\u5b66\u7fd2\u65b9\u6cd5<\/h3>\n\n\n\n<p>RNN\u306f<strong>1\u6587\u5b57\u305a\u3064\u9806\u756a\u306b\u51e6\u7406<\/strong>\u3057\u307e\u3059\u300220\u6587\u5b57\u3092\u4e00\u62ec\u3067\u5165\u308c\u306620\u6587\u5b57\u304c\u51fa\u3066\u304f\u308b\u306e\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u30b9\u30c6\u30c3\u30d71: \u300c\u543e\u300d\u3092\u5165\u529b \u2192 \u300c\u8f29\u300d\u3092\u4e88\u6e2c \u2192 \u640d\u5931L\u2081\u3092\u8a08\u7b97\n             \u2193 \u96a0\u308c\u72b6\u614bh\u2081\u3092\u6b21\u3078\u5f15\u304d\u7d99\u3050\n\u30b9\u30c6\u30c3\u30d72: \u300c\u8f29\u300d\u3092\u5165\u529b \u2192 \u300c\u306f\u300d\u3092\u4e88\u6e2c \u2192 \u640d\u5931L\u2082\u3092\u8a08\u7b97\n             \u2193 \u96a0\u308c\u72b6\u614bh\u2082\u3092\u6b21\u3078\u5f15\u304d\u7d99\u3050\n\u30b9\u30c6\u30c3\u30d73: \u300c\u306f\u300d\u3092\u5165\u529b \u2192 \u300c\u732b\u300d\u3092\u4e88\u6e2c \u2192 \u640d\u5931L\u2083\u3092\u8a08\u7b97\n             \u2193\n          ...\uff0820\u30b9\u30c6\u30c3\u30d7\u7e70\u308a\u8fd4\u3059\uff09\n             \u2193\n\u5168\u30b9\u30c6\u30c3\u30d7\u306e\u640d\u5931\u3092\u5408\u8a08 \u2192 \u30d1\u30e9\u30e1\u30fc\u30bf\u66f4\u65b0\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">\u5165\u529b<\/h4>\n\n\n\n<p>\u5404\u30b9\u30c6\u30c3\u30d7\u3067<strong>1\u6587\u5b57<\/strong>\u3092\u5165\u529b\u3057\u307e\u3059\u3002\u5b66\u7fd2\u6642\u306f\u9577\u305520\u306e\u30b7\u30fc\u30b1\u30f3\u30b9\u3092\u7528\u610f\u3057\u3001\u5148\u982d\u304b\u30891\u6587\u5b57\u305a\u3064\u9806\u756a\u306b\u5165\u529b\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u5165\u529b\uff081\u6587\u5b57\u305a\u3064\u51e6\u7406\uff09:\n\u30b9\u30c6\u30c3\u30d71: \u300c\u543e\u300d\n\u30b9\u30c6\u30c3\u30d72: \u300c\u8f29\u300d\n\u30b9\u30c6\u30c3\u30d73: \u300c\u306f\u300d\n  ...\n\u30b9\u30c6\u30c3\u30d720: \u300c\u751f\u300d\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">\u51fa\u529b<\/h4>\n\n\n\n<p>\u5404\u30b9\u30c6\u30c3\u30d7\u3067**1\u6587\u5b57\u306e\u4e88\u6e2c\u30b9\u30b3\u30a2\uff0874\u6b21\u5143\uff09**\u3092\u51fa\u529b\u3057\u307e\u3059\u3002\u6b63\u89e3\u306f\u5165\u529b\u30921\u6587\u5b57\u5f8c\u308d\u306b\u305a\u3089\u3057\u305f\u3082\u306e\u3067\u3059\u3002\u4eca\u56de\u306f\u6587\u5b57\u6570\u304c74\u306a\u306e\u306774\u6b21\u5143\u306b\u5bfe\u5fdc\u3057\u305f\u78ba\u7387\u3092\u51fa\u529b\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\u30b9\u30c6\u30c3\u30d71\u306e\u6b63\u89e3: \u300c\u8f29\u300d\n\u30b9\u30c6\u30c3\u30d72\u306e\u6b63\u89e3: \u300c\u306f\u300d\n\u30b9\u30c6\u30c3\u30d73\u306e\u6b63\u89e3: \u300c\u732b\u300d\n  ...\n\u30b9\u30c6\u30c3\u30d720\u306e\u6b63\u89e3: \u300c\u308c\u300d\n<\/code><\/pre>\n\n\n\n<p>\u30e2\u30c7\u30eb\u306f\u300c\u73fe\u5728\u306e\u6587\u5b57\u3068\u904e\u53bb\u306e\u96a0\u308c\u72b6\u614b\u304b\u3089\u3001\u6b21\u306e\u6587\u5b57\u3092\u4e88\u6e2c\u3059\u308b\u300d\u30bf\u30b9\u30af\u3068\u3057\u3066\u5b66\u7fd2\u3057\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u30e2\u30c7\u30eb\u5b9a\u7fa9<\/h3>\n\n\n\n<pre class=\"wp-block-code\"><code>class CharRNN(nn.Module):\n    def __init__(self, vocab_size, embed_dim, hidden_dim):\n        super().__init__()\n        self.embedding = nn.Embedding(vocab_size, embed_dim)   # 74 \u2192 32\u6b21\u5143\n        self.rnn = nn.RNN(embed_dim, hidden_dim, batch_first=True)  # 32 \u2192 128\u6b21\u5143\n        self.fc = nn.Linear(hidden_dim, vocab_size)            # 128 \u2192 74\u6b21\u5143\n<\/code><\/pre>\n\n\n\n<p>\u5404\u5c64\u306e\u5f79\u5272\u306f\u4ee5\u4e0b\u306e\u3068\u304a\u308a\u3067\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>\u5c64<\/th><th>\u5165\u529b\u6b21\u5143<\/th><th>\u51fa\u529b\u6b21\u5143<\/th><th>\u5f79\u5272<\/th><\/tr><\/thead><tbody><tr><td>Embedding<\/td><td>74\uff08\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\uff09<\/td><td>32<\/td><td>\u6587\u5b57\u3092\u5bc6\u306a\u30d9\u30af\u30c8\u30eb\u306b\u5909\u63db<\/td><\/tr><tr><td>RNN<\/td><td>32<\/td><td>128<\/td><td>\u96a0\u308c\u72b6\u614b\u3068\u3057\u3066\u6587\u8108\u3092\u84c4\u7a4d<\/td><\/tr><tr><td>Linear<\/td><td>128<\/td><td>74<\/td><td>\u5404\u6587\u5b57\u306e\u30b9\u30b3\u30a2\u3092\u51fa\u529b<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">\u640d\u5931\u63a8\u79fb<\/h3>\n\n\n\n<p>300\u30a8\u30dd\u30c3\u30af\u5b66\u7fd2\u3057\u305f\u969b\u306e\u640d\u5931\u306e\u63a8\u79fb\u3067\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Epoch   1: 4.3357  \uff08\u307b\u307c\u30e9\u30f3\u30c0\u30e0\u306a\u4e88\u6e2c\uff09\nEpoch  50: 0.2062  \uff08\u6025\u901f\u306b\u5b66\u7fd2\u304c\u9032\u3080\uff09\nEpoch 100: 0.1253\nEpoch 150: 0.1045\nEpoch 200: 0.0791  \uff08\u6700\u5c0f\u5024\u4ed8\u8fd1\uff09\nEpoch 250: 0.0996\nEpoch 300: 0.1063\n<\/code><\/pre>\n\n\n\n<p>\u5b66\u7fd2\u521d\u671f\u306b\u640d\u5931\u304c\u6025\u6fc0\u306b\u4e0b\u304c\u308a\u3001\u305d\u306e\u5f8c\u7de9\u3084\u304b\u306b\u53ce\u675f\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">\u751f\u6210\u7d50\u679c<\/h3>\n\n\n\n<p>\u5b66\u7fd2\u6e08\u307f\u30e2\u30c7\u30eb\u306b\u300c\u543e\u300d\u3092\u5165\u529b\u3068\u3057\u3066\u4e0e\u3048\u3001temperature\u3092\u5909\u3048\u3066\u751f\u6210\u3057\u305f\u7d50\u679c\u3067\u3059\u3002<\/p>\n\n\n\n<p><strong>temperature = 0.5\uff08\u4fdd\u5b88\u7684\uff09<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&#091;1] \u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u3002\u540d\u524d\u306f\u307e\u3060\u306a\u3044\u3002\n&#091;2] \u543e\u8f29\u306f\u3053\u3053\u3067\u59cb\u3081\u3066\u4eba\u9593\u3068\u3044\u3046\u3082\u306e\u3092\u898b\u305f\u3002\n&#091;3] \u543e\u8f29\u306f\u8003\u3048\u308b\u732b\u3067\u3042\u308b\u3002\n<\/code><\/pre>\n\n\n\n<p>\u30b3\u30fc\u30d1\u30b9\u306b\u5fe0\u5b9f\u306a\u6587\u7ae0\u304c\u751f\u6210\u3055\u308c\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<p><strong>temperature = 1.0\uff08\u6a19\u6e96\uff09<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&#091;1] \u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u3002\u732b\u306e\u540d\u524d\u306f\u4f55\u3067\u3082\u3044\u3044\u3002\n&#091;2] \u543e\u8f29\u306f\u8003\u3048\u308b\u732b\u3067\u3042\u308b\u3002\n&#091;3] \u543e\u8f29\u306f\u8003\u3048\u308b\u732b\u3067\u3042\u308b\u3002\n<\/code><\/pre>\n\n\n\n<p><strong>temperature = 1.5\uff08\u5275\u9020\u7684\uff09<\/strong><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&#091;1] \u543e\u8f29\u306f\u8003\u3048\u3042\u308b\u3002\u543e\u8f29\u306f\u732b\u3067\u540d\u308b\u3002\n&#091;2] \u543e\u8f29\u306f\u732b\u3067\u3042\u308b\u3002\u4e8b\u3053\u540d\u524d\u3092\u6ce3\u3066\u3066\u3044\u305f\u8003\u3048\u308b\u3002\n&#091;3] \u543e\u8f29\u306f\u3057\u304b\u306f\u732b\u3067\u30cb\u30e3\u30fc\u30cb\u30e3\u30fc\u30cb\u30e3\u30fc\u6ce3\u3044\u3066\u3044\u305f\u4e8b\u3060\u3051\u306f\u8a18\u61b6\u3057\u3066\u3044\u308b\u3002\n<\/code><\/pre>\n\n\n\n<p>temperature\u3092\u4e0a\u3052\u308b\u3068\u6587\u6cd5\u7684\u306b\u5d29\u308c\u305f\u6587\u7ae0\u304c\u51fa\u3066\u304d\u307e\u3059\u3002\u3053\u308c\u306f\u30b3\u30fc\u30d1\u30b9\u304c\u5c11\u306a\u3044\u305f\u3081\u3001\u4f4e\u78ba\u7387\u306e\u9077\u79fb\u304c\u9078\u3070\u308c\u308b\u3068\u672a\u5b66\u7fd2\u306e\u7d44\u307f\u5408\u308f\u305b\u306b\u306a\u308b\u305f\u3081\u3067\u3059\u3002<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n\n\n\n<h2 class=\"wp-block-heading\">\u7d42\u308f\u308a\u306b<\/h2>\n\n\n\n<p>RNN\u306e\u96a0\u308c\u72b6\u614b\u306f\u904e\u53bb\u306e\u5168\u6587\u8108\u3092\u84c4\u7a4d\u3067\u304d\u307e\u3059\u304c\u3001\u300c1\u3064\u306e\u30d9\u30af\u30c8\u30eb\u306b\u5727\u7e2e\u3059\u308b\u300d\u3068\u3044\u3046\u5236\u7d04\u304b\u3089\u9577\u671f\u4f9d\u5b58\u306e\u5b66\u7fd2\u304c\u96e3\u3057\u3044\u3068\u3044\u3046\u554f\u984c\u304c\u3042\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u6b21\u306e\u8a18\u4e8b\u3067\u306f\u3001\u307e\u3063\u305f\u304f\u7570\u306a\u308b\u30a2\u30d7\u30ed\u30fc\u30c1\u3067\u751f\u6210\u30e2\u30c7\u30eb\u3092\u5b9f\u88c5\u3057\u307e\u3059\u3002**VAE\uff08\u5909\u5206\u30aa\u30fc\u30c8\u30a8\u30f3\u30b3\u30fc\u30c0\uff09**\u306f\u6587\u7ae0\u3092\u9010\u6b21\u751f\u6210\u3059\u308b\u306e\u3067\u306f\u306a\u304f\u3001\u6587\u7ae0\u5168\u4f53\u3092\u6f5c\u5728\u7a7a\u9593\u4e0a\u306e\u30d9\u30af\u30c8\u30eb\u306b\u5727\u7e2e\u3057\u3001\u305d\u3053\u304b\u3089\u30c7\u30b3\u30fc\u30c9\u3059\u308b\u65b9\u5f0f\u3067\u3059\u3002\u6f5c\u5728\u7a7a\u9593\u4e0a\u3067\u306e\u88dc\u9593\u306a\u3069\u3001RNN\u306b\u306f\u3067\u304d\u306a\u3044\u64cd\u4f5c\u304c\u53ef\u80fd\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u306f\u3058\u3081\u306b \u751f\u6210\u30e2\u30c7\u30eb\u3068\u306f \u751f\u6210\u30e2\u30c7\u30eb\u3068\u306f\u3001\u30c7\u30fc\u30bf\u306e\u5206\u5e03\u3092\u5b66\u7fd2\u3057\u3001\u65b0\u3057\u3044\u30c7\u30fc\u30bf\u3092\u751f\u6210\u3059\u308b\u30e2\u30c7\u30eb\u3067\u3059\u3002\u524d\u56de\u306e\u8a18\u4e8b\u3067\u306f\u305d\u306e\u6700\u3082\u30b7\u30f3\u30d7\u30eb\u306a\u5b9f\u88c5\u3068\u3057\u3066\u30de\u30eb\u30b3\u30d5\u9023\u9396\u3092\u53d6\u308a\u4e0a\u3052\u307e\u3057\u305f\u3002 \u30de\u30eb\u30b3\u30d5\u9023\u9396\u3068\u306e\u95a2\u4fc2 \u7c21\u5358\u306a\u5fa9\u7fd2 \u30de\u30eb\u30b3\u30d5\u9023\u9396\u306f [&hellip;]<\/p>\n","protected":false},"author":44,"featured_media":2525,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","swell_btn_cv_data":"","footnotes":"","_wp_rev_ctl_limit":""},"categories":[1249],"tags":[96,1399,553,1284],"class_list":["post-9649","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-knowledge","tag-ai","tag-rnn","tag-553","tag-1284"],"_links":{"self":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/9649","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/users\/44"}],"replies":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/comments?post=9649"}],"version-history":[{"count":2,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/9649\/revisions"}],"predecessor-version":[{"id":9654,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/9649\/revisions\/9654"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media\/2525"}],"wp:attachment":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media?parent=9649"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/categories?post=9649"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/tags?post=9649"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}