{"id":10363,"date":"2026-11-06T18:15:14","date_gmt":"2026-11-06T09:15:14","guid":{"rendered":"https:\/\/since2020.jp\/media\/?p=10363"},"modified":"2026-11-06T18:15:14","modified_gmt":"2026-11-06T09:15:14","slug":"whatskl","status":"publish","type":"post","link":"https:\/\/since2020.jp\/media\/whatskl\/","title":{"rendered":"KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u3068\u306f\u4f55\u304b: \u30b3\u30a4\u30f3\u6295\u3052\u3067\u78ba\u304b\u3081\u308b\u300c\u5206\u5e03\u306e\u305a\u308c\u300d"},"content":{"rendered":"\n

\u516c\u5e73\u306a\u30b3\u30a4\u30f3 P<\/mi>P<\/annotation><\/semantics><\/math>\uff08\u8868\u306e\u78ba\u7387 0.5\uff09\u3068\u3001\u8868\u304c\u51fa\u3084\u3059\u3044\u30b3\u30a4\u30f3 Q<\/mi>Q<\/annotation><\/semantics><\/math>\uff08\u8868\u306e\u78ba\u7387 0.8\uff09\u3092\u8003\u3048\u308b\u3002Q<\/mi>Q<\/annotation><\/semantics><\/math> \u304c\u672c\u7269\u306e\u30b3\u30a4\u30f3\u3060\u3068\u4fe1\u3058\u305f\u307e\u307e\u3001\u5b9f\u969b\u306b\u306f P<\/mi>P<\/annotation><\/semantics><\/math> \u304b\u3089\u51fa\u305f\u7d50\u679c\u3092\u898b\u308b\u3068\u3057\u305f\u3089\u3001\u3069\u308c\u304f\u3089\u3044\u300c\u4e88\u60f3\u5916\u300d\u306b\u611f\u3058\u308b\u3060\u308d\u3046\u304b\u3002<\/p>\n\n\n\n

\u516c\u5e73\u306a\u30b3\u30a4\u30f3 P \u3068\u504f\u3063\u305f\u30b3\u30a4\u30f3 Q \u306e\u78ba\u7387\u306e\u6bd4\u8f03<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

\u56f31: \u516c\u5e73\u306a\u30b3\u30a4\u30f3 P\uff08\u8868\u306e\u78ba\u7387 0.5\uff09\u3068\u504f\u3063\u305f\u30b3\u30a4\u30f3 Q\uff08\u8868\u306e\u78ba\u7387 0.8\uff09\u304c\u3001\u88cf\u3068\u8868\u305d\u308c\u305e\u308c\u306b\u5272\u308a\u5f53\u3066\u308b\u78ba\u7387\u3092\u4e26\u3079\u305f\u68d2\u30b0\u30e9\u30d5\u3002<\/em><\/p>\n\n\n\n

\u56f31\u306f2\u3064\u306e\u5206\u5e03\u3092\u4e26\u3079\u305f\u3082\u306e\u3067\u3042\u308b\u3002\u305a\u308c\u306f\u76ee\u3067\u898b\u3066\u5206\u304b\u308b\u304c\u3001\u307e\u30601\u3064\u306e\u6570\u3068\u3057\u3066\u306f\u8868\u305b\u3066\u3044\u306a\u3044\u3002<\/p>\n\n\n\n

\u77e5\u308a\u305f\u3044\u306e\u306f\u6b21\u306e3\u70b9\u3067\u3042\u308b\u3002<\/p>\n\n\n\n

    \n
  • \u305a\u308c\u306f\u3069\u3046\u5b9a\u7fa9\u3067\u304d\u308b\u306e\u304b\u3002<\/li>\n\n\n\n
  • \u305a\u308c\u306e\u6027\u8cea\uff11\uff08\u306a\u305c\u8ca0\u306b\u306a\u3089\u306a\u3044\u306e\u304b\u3002\uff09<\/li>\n\n\n\n
  • \u305a\u308c\u306e\u6027\u8cea\uff12\uff08\u306a\u305c P<\/mi>P<\/annotation><\/semantics><\/math> \u3068 Q<\/mi>Q<\/annotation><\/semantics><\/math> \u3092\u5165\u308c\u66ff\u3048\u308b\u3068\u5024\u304c\u5909\u308f\u308b\u306e\u304b\u3002\uff09<\/li>\n<\/ul>\n\n\n\n

    \u8a2d\u5b9a<\/h2>\n\n\n\n
    \n

    \u7528\u8a9e\u306e\u88dc\u8db3<\/strong><\/p>\n\n\n\n

      \n
    • \u78ba\u7387\u5206\u5e03: \u5404\u7d50\u679c\u306e\u8d77\u3053\u308a\u3084\u3059\u3055\u3092\u4e26\u3079\u305f\u3082\u306e\u3002\u30b3\u30a4\u30f3\u306a\u3089 P<\/mi>=<\/mo>(<\/mo>p<\/mi>,<\/mo><\/mspace>1<\/mn>\u2212<\/mo>p<\/mi>)<\/mo><\/mrow>P=(p,\\,1-p)<\/annotation><\/semantics><\/math>\u3002<\/li>\n\n\n\n
    • P<\/mi>P<\/annotation><\/semantics><\/math> \u306f\u771f\u306e\u5206\u5e03\u3001Q<\/mi>Q<\/annotation><\/semantics><\/math> \u306f\u30e2\u30c7\u30eb\uff08\u3053\u3061\u3089\u304c\u4fe1\u3058\u3066\u3044\u308b\uff09\u5206\u5e03\u3002Q<\/mi>Q<\/annotation><\/semantics><\/math> \u306e\u8868\u306e\u78ba\u7387\u3092 q<\/mi>q<\/annotation><\/semantics><\/math> \u3068\u3059\u308b\u3002<\/li>\n\n\n\n
    • p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>p(x)<\/annotation><\/semantics><\/math>\u3001q<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>q(x)<\/annotation><\/semantics><\/math>: \u7d50\u679c x<\/mi>x<\/annotation><\/semantics><\/math> \u306b\u5bfe\u3057\u3066\u3001P<\/mi>P<\/annotation><\/semantics><\/math> \u3068 Q<\/mi>Q<\/annotation><\/semantics><\/math> \u304c\u305d\u308c\u305e\u308c\u5272\u308a\u5f53\u3066\u308b\u78ba\u7387\u3002\u30b3\u30a4\u30f3\u3067\u306f x<\/mi>x<\/annotation><\/semantics><\/math> \u306f\u8868\u304b\u88cf\u3067\u3042\u308b\u3002<\/li>\n\n\n\n
    • \u9a5a\u304d: \u7d50\u679c x<\/mi>x<\/annotation><\/semantics><\/math> \u306b\u5bfe\u3059\u308b \u2212<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>q<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>-\\log q(x)<\/annotation><\/semantics><\/math>\u3002\u3081\u3063\u305f\u306b\u8d77\u304d\u306a\u3044\u3053\u3068\u307b\u3069\u5927\u304d\u3044\u3002\u4f8b\u3048\u3070\u3001\u78ba\u7387 0.01 \u306e\u51fa\u6765\u4e8b\u306f\u78ba\u7387 0.5 \u306e\u51fa\u6765\u4e8b\u3088\u308a\u9a5a\u304d\u304c\u5927\u304d\u3044\u3002<\/li>\n\n\n\n
    • \u5bfe\u6570\u306f\u81ea\u7136\u5bfe\u6570\u3067\u3001\u5358\u4f4d\u306f nat\uff08\u30ca\u30c3\u30c8\uff09\u3002<\/li>\n<\/ul>\n<\/blockquote>\n\n\n\n

      p<\/mi>(<\/mo>x<\/mi>)<\/mo>><\/mo>0<\/mn><\/mrow>p(x)>0<\/annotation><\/semantics><\/math> \u3068\u306a\u308b\u3059\u3079\u3066\u306e\u7d50\u679c\u3067 q<\/mi>(<\/mo>x<\/mi>)<\/mo>><\/mo>0<\/mn><\/mrow>q(x)>0<\/annotation><\/semantics><\/math> \u3067\u3042\u308b\u3068\u4eee\u5b9a\u3059\u308b\u3002\u3053\u306e\u3068\u304d\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\uff08\u30ab\u30eb\u30d0\u30c3\u30af\u30fb\u30e9\u30a4\u30d6\u30e9\u30fc\u60c5\u5831\u91cf\u3002\u5206\u5e03\u306e\u305a\u308c\uff09\u3092\u6b21\u306e\u3088\u3046\u306b\u5b9a\u7fa9\u3059\u308b [1]\u3002<\/p>\n\n\n\n


      D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>=<\/mo>\u2211<\/mo>x<\/mi><\/munder><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mspace>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>q<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow><\/mfrac><\/mstyle><\/mrow>D_{\\mathrm{KL}}(P\\|Q)=\\displaystyle\\sum_x p(x)\\log\\frac{p(x)}{q(x)}<\/annotation><\/semantics><\/math>
      <\/p>\n\n\n\n

      KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306e\u5b9a\u7fa9\u306e\u6d41\u308c\u56f3\u3092\u4ee5\u4e0b\u306b\u793a\u3059\u3002<\/p>\n\n\n\n

      \"\"<\/figure>\n\n\n\n

      \u56f32: \u771f\u306e\u5206\u5e03 P \u3068\u8fd1\u4f3c\u3059\u308b\u5206\u5e03 Q \u304b\u3089\u7d50\u679c\u3054\u3068\u306e\u9a5a\u304d\u306e\u5dee\u3092\u4f5c\u308a\u3001P \u306e\u3082\u3068\u3067\u5e73\u5747\u3059\u308b\u3068 KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306b\u306a\u308b\u3001\u3068\u3044\u3046\u6d41\u308c\u3092\u793a\u3057\u305f\u6982\u5ff5\u56f3\u3002<\/em><\/p>\n\n\n\n

      \u5c0e\u51fa<\/h2>\n\n\n\n

      \u7d50\u8ad6\u3092\u5148\u306b\u8ff0\u3079\u308b\u3068\u3001D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>D_{\\mathrm{KL}}<\/annotation><\/semantics><\/math> \u306f\u300cQ<\/mi>Q<\/annotation><\/semantics><\/math> \u3092\u57fa\u6e96\u306b\u3057\u305f\u5e73\u5747\u306e\u9a5a\u304d\u300d\u304b\u3089\u300cP<\/mi>P<\/annotation><\/semantics><\/math> \u3092\u57fa\u6e96\u306b\u3057\u305f\u5e73\u5747\u306e\u9a5a\u304d\u300d\u3092\u5f15\u3044\u305f\u91cf\u3067\u3042\u308a\u30010 \u4ee5\u4e0a\u306b\u306a\u308b\u3002<\/p>\n\n\n\n

      \u4ee5\u4e0b\u3001\u548c\u306f p<\/mi>(<\/mo>x<\/mi>)<\/mo>><\/mo>0<\/mn><\/mrow>p(x)>0<\/annotation><\/semantics><\/math> \u3068\u306a\u308b\u7d50\u679c\u306e\u96c6\u5408 S<\/mi>=<\/mo>{<\/mo>x<\/mi>|<\/mo>p<\/mi>(<\/mo>x<\/mi>)<\/mo>><\/mo>0<\/mn>}<\/mo><\/mrow>S=\\{x\\mid p(x)>0\\}<\/annotation><\/semantics><\/math> \u306b\u3064\u3044\u3066\u53d6\u308b\u3002p<\/mi>(<\/mo>x<\/mi>)<\/mo>=<\/mo>0<\/mn><\/mrow>p(x)=0<\/annotation><\/semantics><\/math> \u306e\u9805\u306f 0<\/mn>\u22c5<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>0<\/mn>=<\/mo>0<\/mn><\/mrow>0\\cdot\\log 0=0<\/annotation><\/semantics><\/math> \u3068\u307f\u306a\u3057\u3066\u9664\u3051\u308b\u305f\u3081\u3067\u3042\u308b\u3002<\/p>\n\n\n\n

      \u30b9\u30c6\u30c3\u30d71: \u5bfe\u6570\u3092\u5206\u3051\u308b<\/h3>\n\n\n\n

      log<\/mi>\u2061<\/mo><\/mspace><\/mrow>p<\/mi>q<\/mi><\/mfrac>=<\/mo>\u2212<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>q<\/mi>\u2212<\/mo>(<\/mo>\u2212<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>p<\/mi>)<\/mo><\/mrow>\\log\\frac{p}{q}=-\\log q-(-\\log p)<\/annotation><\/semantics><\/math> \u306a\u306e\u3067\u3001\u6b21\u306e\u3088\u3046\u306b\u66f8\u3051\u308b\u3002<\/p>\n\n\n\n


      D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>=<\/mo>\u2211<\/mo>x<\/mi>\u2208<\/mo>S<\/mi><\/mrow><\/munder><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo>(<\/mo>\u2212<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>q<\/mi>(<\/mo>x<\/mi>)<\/mo>)<\/mo>\u2212<\/mo>\u2211<\/mo>x<\/mi>\u2208<\/mo>S<\/mi><\/mrow><\/munder><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo>(<\/mo>\u2212<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo>)<\/mo><\/mstyle>\\displaystyle D_{\\mathrm{KL}}(P\\|Q)=\\sum_{x\\in S} p(x)\\bigl(-\\log q(x)\\bigr)-\\sum_{x\\in S} p(x)\\bigl(-\\log p(x)\\bigr)<\/annotation><\/semantics><\/math>
      <\/p>\n\n\n\n

      \u30b9\u30c6\u30c3\u30d72: \u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3068\u306e\u95a2\u4fc2<\/h3>\n\n\n\n

      \u7b2c1\u9805\u3092\u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc H<\/mi>(<\/mo>P<\/mi>,<\/mo>Q<\/mi>)<\/mo><\/mrow>H(P,Q)<\/annotation><\/semantics><\/math>\u3001\u7b2c2\u9805\u3092\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc H<\/mi>(<\/mo>P<\/mi>)<\/mo><\/mrow>H(P)<\/annotation><\/semantics><\/math> \u3068\u547c\u3076\u3002<\/p>\n\n\n\n


      D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>=<\/mo>H<\/mi>(<\/mo>P<\/mi>,<\/mo>Q<\/mi>)<\/mo>\u2212<\/mo>H<\/mi>(<\/mo>P<\/mi>)<\/mo><\/mrow>D_{\\mathrm{KL}}(P\\|Q)=H(P,Q)-H(P)<\/annotation><\/semantics><\/math>
      <\/p>\n\n\n\n

      H<\/mi>(<\/mo>P<\/mi>)<\/mo><\/mrow>H(P)<\/annotation><\/semantics><\/math> \u306f Q<\/mi>Q<\/annotation><\/semantics><\/math> \u306b\u4f9d\u5b58\u3057\u306a\u3044\u3002\u3057\u305f\u304c\u3063\u3066\u3001Q<\/mi>Q<\/annotation><\/semantics><\/math> \u3092\u52d5\u304b\u3057\u3066 H<\/mi>(<\/mo>P<\/mi>,<\/mo>Q<\/mi>)<\/mo><\/mrow>H(P,Q)<\/annotation><\/semantics><\/math> \u3092\u6700\u5c0f\u306b\u3059\u308b\u3053\u3068\u3068\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u3092\u6700\u5c0f\u306b\u3059\u308b\u3053\u3068\u306f\u540c\u3058\u554f\u984c\u3067\u3042\u308b\u3002\u6700\u5c24\u63a8\u5b9a\u304c\u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u306e\u6700\u5c0f\u5316\u3068\u540c\u3058\u554f\u984c\u306b\u306a\u308b\u306e\u306f\u3001\u3053\u306e\u305f\u3081\u3067\u3042\u308b\u3002<\/p>\n\n\n\n

      \u30b9\u30c6\u30c3\u30d73: \u975e\u8ca0\u6027<\/h3>\n\n\n\n

      \u307e\u305a\u6570\u5024\u4f8b\u3067\u78ba\u304b\u3081\u308b\u3002p<\/mi>=<\/mo>0.5<\/mn>,<\/mo> <\/mtext>q<\/mi>=<\/mo>0.8<\/mn><\/mrow>p=0.5,\\ q=0.8<\/annotation><\/semantics><\/math> \u306e\u3068\u304d q<\/mi>p<\/mi><\/mfrac>=<\/mo>1.6<\/mn><\/mrow>\\frac{q}{p}=1.6<\/annotation><\/semantics><\/math> \u3067\u3001log<\/mi>\u2061<\/mo><\/mspace><\/mrow>1.6<\/mn>\u2248<\/mo>0.47<\/mn>\u2264<\/mo>0.6<\/mn>=<\/mo>1.6<\/mn>\u2212<\/mo>1<\/mn><\/mrow>\\log 1.6\\approx0.47\\le 0.6=1.6-1<\/annotation><\/semantics><\/math> \u3068\u306a\u308a\u3001\u4e0d\u7b49\u5f0f log<\/mi>\u2061<\/mo><\/mspace><\/mrow>t<\/mi>\u2264<\/mo>t<\/mi>\u2212<\/mo>1<\/mn><\/mrow>\\log t\\le t-1<\/annotation><\/semantics><\/math> \u304c\u6210\u308a\u7acb\u3063\u3066\u3044\u308b\u3002<\/p>\n\n\n\n
      \n

      \u7528\u8a9e\u306e\u88dc\u8db3<\/strong><\/p>\n\n\n\n

        \n
      • \u51f9\u95a2\u6570: \u30b0\u30e9\u30d5\u4e0a\u306e\u4efb\u610f\u306e2\u70b9\u3092\u7d50\u3076\u7dda\u5206\u304c\u3001\u30b0\u30e9\u30d5\u3088\u308a\u4e0b\u306b\u3042\u308b\u95a2\u6570\u3002\u5bfe\u6570\u304c\u305d\u306e\u4f8b\u3067\u3042\u308b\u3002<\/li>\n<\/ul>\n<\/blockquote>\n\n\n\n

        y<\/mi>=<\/mo>t<\/mi>\u2212<\/mo>1<\/mn><\/mrow>y=t-1<\/annotation><\/semantics><\/math> \u306f y<\/mi>=<\/mo>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>t<\/mi><\/mrow>y=\\log t<\/annotation><\/semantics><\/math> \u306e t<\/mi>=<\/mo>1<\/mn><\/mrow>t=1<\/annotation><\/semantics><\/math> \u306b\u304a\u3051\u308b\u63a5\u7dda\u3067\u3042\u308b\u3002\u5bfe\u6570\u306f\u51f9\u95a2\u6570\u306a\u306e\u3067\u3001\u30b0\u30e9\u30d5\u306f\u63a5\u7dda\u306e\u4e0b\u5074\u306b\u3042\u308b\u3002\u3053\u308c\u304c log<\/mi>\u2061<\/mo><\/mspace><\/mrow>t<\/mi>\u2264<\/mo>t<\/mi>\u2212<\/mo>1<\/mn><\/mrow>\\log t\\le t-1<\/annotation><\/semantics><\/math> \u306e\u7406\u7531\u3067\u3042\u308b [2]\u3002<\/p>\n\n\n\n

        S<\/mi>S<\/annotation><\/semantics><\/math> \u306e\u5404 x<\/mi>x<\/annotation><\/semantics><\/math> \u3067 t<\/mi>=<\/mo>q<\/mi>(<\/mo>x<\/mi>)<\/mo>\/<\/mi>p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>t=q(x)\/p(x)<\/annotation><\/semantics><\/math> \u3068\u304a\u304f\u3002p<\/mi>(<\/mo>x<\/mi>)<\/mo>><\/mo>0<\/mn><\/mrow>p(x)>0<\/annotation><\/semantics><\/math> \u306a\u306e\u3067\u3001\u3053\u306e\u6bd4\u306f\u5b9a\u7fa9\u3067\u304d\u308b\u3002\u3059\u308b\u3068\u6b21\u306e\u3088\u3046\u306b\u306a\u308b [3]\u3002<\/p>\n\n\n\n


        \u2212<\/mo>D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>=<\/mo>\u2211<\/mo>x<\/mi>\u2208<\/mo>S<\/mi><\/mrow><\/munder><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mspace>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>q<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow><\/mfrac>\u2264<\/mo>\u2211<\/mo>x<\/mi>\u2208<\/mo>S<\/mi><\/mrow><\/munder><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo>(<\/mo>q<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow>p<\/mi>(<\/mo>x<\/mi>)<\/mo><\/mrow><\/mfrac>\u2212<\/mo>1<\/mn>)<\/mo>=<\/mo>\u2211<\/mo>x<\/mi>\u2208<\/mo>S<\/mi><\/mrow><\/munder><\/mrow>q<\/mi>(<\/mo>x<\/mi>)<\/mo>\u2212<\/mo>1<\/mn>\u2264<\/mo>0<\/mn><\/mstyle><\/mrow>-D_{\\mathrm{KL}}(P\\|Q)=\\displaystyle\\sum_{x\\in S} p(x)\\log\\frac{q(x)}{p(x)}\\le\\sum_{x\\in S} p(x)\\Bigl(\\frac{q(x)}{p(x)}-1\\Bigr)=\\sum_{x\\in S} q(x)-1\\le 0<\/annotation><\/semantics><\/math>
        <\/p>\n\n\n\n

        \u6700\u5f8c\u306e\u4e0d\u7b49\u53f7\u304c\u6210\u308a\u7acb\u3064\u306e\u306f\u3001S<\/mi>S<\/annotation><\/semantics><\/math> \u4e0a\u3067\u306e q<\/mi>q<\/annotation><\/semantics><\/math> \u306e\u548c\u304c\u3001\u5168\u4f53\u306e\u548c\u3067\u3042\u308b 1 \u3092\u8d85\u3048\u306a\u3044\u304b\u3089\u3067\u3042\u308b\u3002<\/p>\n\n\n\n

        \u3088\u3063\u3066 D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>\u2265<\/mo>0<\/mn><\/mrow>D_{\\mathrm{KL}}(P\\|Q)\\ge0<\/annotation><\/semantics><\/math> \u3067\u3042\u308a\u3001\u7b49\u53f7\u304c\u6210\u308a\u7acb\u3064\u306e\u306f P<\/mi>=<\/mo>Q<\/mi><\/mrow>P=Q<\/annotation><\/semantics><\/math> \u306e\u3068\u304d\u306b\u9650\u308b [4]\u3002<\/p>\n\n\n\n
        \n

        \u4f59\u8ac7<\/strong> \u7b49\u53f7\u6761\u4ef6\u306f\u3001log<\/mi>\u2061<\/mo><\/mspace><\/mrow>t<\/mi>=<\/mo>t<\/mi>\u2212<\/mo>1<\/mn><\/mrow>\\log t=t-1<\/annotation><\/semantics><\/math> \u304c t<\/mi>=<\/mo>1<\/mn><\/mrow>t=1<\/annotation><\/semantics><\/math> \u306e\u3068\u304d\u3060\u3051\u6210\u308a\u7acb\u3064\u3053\u3068\u3001\u304a\u3088\u3073 S<\/mi>S<\/annotation><\/semantics><\/math> \u306e\u5916\u3067 q<\/mi>=<\/mo>0<\/mn><\/mrow>q=0<\/annotation><\/semantics><\/math> \u3068\u306a\u308b\u3053\u3068\u304b\u3089\u5f93\u3046\u3002\u53b3\u5bc6\u306a\u8b70\u8ad6\u306f\u7701\u7565\u3002<\/p>\n<\/blockquote>\n\n\n\n

        \u30b9\u30c6\u30c3\u30d74: \u975e\u5bfe\u79f0\u6027<\/h3>\n\n\n\n

        \u30b3\u30a4\u30f3\u3067\u306f\u3001\u6b21\u306e2\u3064\u306e\u5f0f\u306f\u4e00\u822c\u306b\u7570\u306a\u308b\u5024\u306b\u306a\u308b\u3002<\/p>\n\n\n\n


        D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>=<\/mo>p<\/mi><\/mspace>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>p<\/mi>q<\/mi><\/mfrac>+<\/mo>(<\/mo>1<\/mn>\u2212<\/mo>p<\/mi>)<\/mo><\/mspace>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>1<\/mn>\u2212<\/mo>p<\/mi><\/mrow>1<\/mn>\u2212<\/mo>q<\/mi><\/mrow><\/mfrac>,<\/mo><\/mrow><\/mtd><\/mtr>D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>Q<\/mi>\u2016<\/mi>P<\/mi>)<\/mo>=<\/mo>q<\/mi><\/mspace>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>q<\/mi>p<\/mi><\/mfrac>+<\/mo>(<\/mo>1<\/mn>\u2212<\/mo>q<\/mi>)<\/mo><\/mspace>log<\/mi>\u2061<\/mo><\/mspace><\/mrow>1<\/mn>\u2212<\/mo>q<\/mi><\/mrow>1<\/mn>\u2212<\/mo>p<\/mi><\/mrow><\/mfrac><\/mrow><\/mtd><\/mtr><\/mtable>\\begin{aligned} D_{\\mathrm{KL}}(P\\|Q)=p\\log\\frac pq+(1-p)\\log\\frac{1-p}{1-q},\\\\ D_{\\mathrm{KL}}(Q\\|P)=q\\log\\frac qp+(1-q)\\log\\frac{1-q}{1-p} \\end{aligned}<\/annotation><\/semantics><\/math>
        <\/p>\n\n\n\n

        \u5e73\u5747\u3092\u53d6\u308b\u5206\u5e03\u304c P<\/mi>P<\/annotation><\/semantics><\/math> \u304b Q<\/mi>Q<\/annotation><\/semantics><\/math> \u304b\u3067\u3001\u5404\u7d50\u679c\u306b\u4ed8\u304f\u91cd\u307f\u304c\u5909\u308f\u308b\u305f\u3081\u3067\u3042\u308b\u3002<\/p>\n\n\n\n

        \u6570\u5024\u5b9f\u9a13<\/h2>\n\n\n\n

        \u6b21\u306e\u30b9\u30af\u30ea\u30d7\u30c8\u3067\u30014\u3064\u306e\u4e3b\u5f35\u3092\u78ba\u304b\u3081\u308b\u3002<\/p>\n\n\n\n

          \n
        1. q<\/mi>=<\/mo>0.5<\/mn><\/mrow>q=0.5<\/annotation><\/semantics><\/math> \u306e\u3068\u304d\u3001\u6700\u5c0f\u5024 0 \u306b\u306a\u308b\u3002<\/li>\n\n\n\n
        2. \u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u304b\u3089\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3092\u5f15\u304f\u3068\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306b\u4e00\u81f4\u3059\u308b\u3002<\/li>\n\n\n\n
        3. \u5411\u304d\u3092\u5165\u308c\u66ff\u3048\u308b\u3068\u5024\u304c\u5909\u308f\u308b\u3002<\/li>\n\n\n\n
        4. \u7d4c\u9a13\u5206\u5e03\u306e KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306f\u3001\u6295\u3052\u308b\u56de\u6570 N<\/mi>N<\/annotation><\/semantics><\/math> \u304c\u5897\u3048\u308b\u3068 0 \u306b\u8fd1\u3065\u304f\u3002<\/li>\n<\/ol>\n\n\n\n

          \u5148\u306b\u3001\u5b9f\u9a13\u3067\u4f7f\u3046\u7528\u8a9e\u3092\u6574\u7406\u3057\u3066\u304a\u304f\u3002<\/p>\n\n\n\n

          \n

          \u7528\u8a9e\u306e\u88dc\u8db3<\/strong><\/p>\n\n\n\n

            \n
          • \u30d9\u30eb\u30cc\u30fc\u30a4\u5206\u5e03 <\/mtext>Bernoulli<\/mi><\/mrow>(<\/mo>q<\/mi>)<\/mo><\/mrow>\\mathrm{Bernoulli}(q)<\/annotation><\/semantics><\/math>: \u8868\u304c\u78ba\u7387 q<\/mi>q<\/annotation><\/semantics><\/math>\u3001\u88cf\u304c\u78ba\u7387 1<\/mn>\u2212<\/mo>q<\/mi><\/mrow>1-q<\/annotation><\/semantics><\/math> \u3067\u51fa\u308b\u30b3\u30a4\u30f3\u306e\u5206\u5e03\u3002<\/li>\n\n\n\n
          • \u7d4c\u9a13\u5206\u5e03: \u5b9f\u969b\u306b\u6295\u3052\u305f\u7d50\u679c\u304b\u3089\u6c42\u3081\u305f\u8868\u306e\u5272\u5408\u3092\u3001\u8868\u306e\u78ba\u7387\u3068\u307f\u306a\u3057\u305f\u5206\u5e03\u3002<\/li>\n\n\n\n
          • \u4e71\u6570\u30b7\u30fc\u30c9: \u4e71\u6570\u306e\u521d\u671f\u5024\u3002\u56fa\u5b9a\u3059\u308b\u3068\u3001\u4f55\u5ea6\u5b9f\u884c\u3057\u3066\u3082\u540c\u3058\u7d50\u679c\u306b\u306a\u308b\u3002<\/li>\n\n\n\n
          • \u4e21\u5bfe\u6570\u8ef8: \u6a2a\u8ef8\u3082\u7e26\u8ef8\u3082\u5bfe\u6570\u76ee\u76db\u308a\u306b\u3057\u305f\u30b0\u30e9\u30d5\u3002\u6841\u3092\u307e\u305f\u3050\u5024\u3092\u6bd4\u3079\u3084\u3059\u3044\u3002<\/li>\n<\/ul>\n<\/blockquote>\n\n\n\n

            python<\/p>\n\n\n\n

            import numpy as np\nfrom scipy.special import xlogy, xlog1py\n\n\ndef kl(p, q):\n    """KL(Bernoulli(p) || Bernoulli(q)) in nats. 0*log0 is handled by xlogy."""\n    return xlogy(p, p \/ q) + xlogy(1 - p, (1 - p) \/ (1 - q))\n\n\n# (1) P = Bernoulli(0.5), Q = Bernoulli(q): KL(P||Q) as a function of q\np = 0.5\nqs = np.linspace(0.01, 0.99, 99)\nkl_pq = kl(p, qs)\nprint(f"min of KL(P||Q) at q ={qs[np.argmin(kl_pq)]:.2f}, value ={kl_pq.min():.2e}")\n\n# (2) cross entropy - entropy = KL\ncross_ent = -(xlogy(p, qs) + xlog1py(1 - p, -qs))\nentropy_p = -(xlogy(p, p) + xlogy(1 - p, 1 - p))\nkl_diff = cross_ent - entropy_p\nprint(f"max |(H(P,Q) - H(P)) - KL| ={np.max(np.abs(kl_diff - kl_pq)):.2e}")\n\n# (3) asymmetry: P fixed at 0.8, Q = Bernoulli(q)\np2 = 0.8\nkl_fwd = kl(p2, qs)   # KL(P||Q)\nkl_rev = kl(qs, p2)   # KL(Q||P)\nprint(f"p=0.5, q=0.8: KL(P||Q) ={kl(0.5, 0.8):.4f}, KL(Q||P) ={kl(0.8, 0.5):.4f}")\n\n# (4) empirical distribution of N tosses of the true coin (nested samples)\nrng = np.random.default_rng(0)\nNs = np.array([10, 100, 1000, 10000, 100000])\nflips = rng.random(Ns.max()) < p\nphat = np.array([flips[:n].mean() for n in Ns])\nkl_hat = kl(phat, p)  # KL(empirical || true coin)\nfor n, v in zip(Ns, kl_hat):\n    print(f"N ={n:6d}: KL(empirical||true) ={v:.2e}")<\/code><\/pre><\/div>\n\n\n\n
            output<\/p>\n\n\n\n
            min of KL(P||Q) at q = 0.50, value = 0.00e+00\nmax |(H(P,Q) - H(P)) - KL| = 4.44e-16\np=0.5, q=0.8: KL(P||Q) = 0.2231, KL(Q||P) = 0.1927\nN =     10: KL(empirical||true) = 8.23e-02\nN =    100: KL(empirical||true) = 7.22e-03\nN =   1000: KL(empirical||true) = 1.46e-03\nN =  10000: KL(empirical||true) = 2.00e-06\nN = 100000: KL(empirical||true) = 1.92e-06<\/code><\/pre><\/div>\n\n\n\n
            \u51fa\u529b\u304b\u3089\u8aad\u307f\u53d6\u308c\u308b\u3053\u3068\u306f\u6b21\u306e\u3068\u304a\u308a\u3067\u3042\u308b\u3002<\/p>\n\n\n\n
              \n
            • (1) \u6700\u5c0f\u306f q<\/mi>=<\/mo>0.50<\/mn><\/mrow>q=0.50<\/annotation><\/semantics><\/math> \u3067\u3001\u5024\u306f 0 \u3060\u3063\u305f\u3002<\/li>\n\n\n\n
            • (2) H<\/mi>(<\/mo>P<\/mi>,<\/mo>Q<\/mi>)<\/mo>\u2212<\/mo>H<\/mi>(<\/mo>P<\/mi>)<\/mo><\/mrow>H(P,Q)-H(P)<\/annotation><\/semantics><\/math> \u3068 KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306e\u5dee\u306f\u3001\u6700\u5927\u3067\u3082 4.44<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>16<\/mn><\/mrow><\/msup><\/mrow>4.44\\times10^{-16}<\/annotation><\/semantics><\/math> \u3067\u3001\u4e38\u3081\u8aa4\u5dee\u306e\u7bc4\u56f2\u3067\u4e00\u81f4\u3057\u3066\u3044\u308b\u3002<\/li>\n\n\n\n
            • (3) p<\/mi>=<\/mo>0.5<\/mn>,<\/mo> <\/mtext>q<\/mi>=<\/mo>0.8<\/mn><\/mrow>p=0.5,\\ q=0.8<\/annotation><\/semantics><\/math> \u3067\u306f D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mi>Q<\/mi>)<\/mo>=<\/mo>0.2231<\/mn><\/mrow>D_{\\mathrm{KL}}(P\\|Q)=0.2231<\/annotation><\/semantics><\/math>\u3001D<\/mi><\/mtext>KL<\/mi><\/mrow><\/msub>(<\/mo>Q<\/mi>\u2016<\/mi>P<\/mi>)<\/mo>=<\/mo>0.1927<\/mn><\/mrow>D_{\\mathrm{KL}}(Q\\|P)=0.1927<\/annotation><\/semantics><\/math> \u3068\u306a\u308a\u3001\u5411\u304d\u3092\u5165\u308c\u66ff\u3048\u308b\u3068\u5024\u304c\u5909\u308f\u308b\u3002<\/li>\n\n\n\n
            • (4) N<\/mi>=<\/mo>10<\/mn><\/mrow>N=10<\/annotation><\/semantics><\/math> \u3067\u306f 8.23<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>2<\/mn><\/mrow><\/msup><\/mrow>8.23\\times10^{-2}<\/annotation><\/semantics><\/math>\u3001N<\/mi>=<\/mo>100<\/mn><\/mrow>N=100<\/annotation><\/semantics><\/math> \u3067\u306f 7.22<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>3<\/mn><\/mrow><\/msup><\/mrow>7.22\\times10^{-3}<\/annotation><\/semantics><\/math>\u3001N<\/mi>=<\/mo>1000<\/mn><\/mrow>N=1000<\/annotation><\/semantics><\/math> \u3067\u306f 1.46<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>3<\/mn><\/mrow><\/msup><\/mrow>1.46\\times10^{-3}<\/annotation><\/semantics><\/math> \u3068\u6e1b\u308a\u3001N<\/mi>=<\/mo>100000<\/mn><\/mrow>N=100000<\/annotation><\/semantics><\/math> \u3067\u306f 1.92<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>6<\/mn><\/mrow><\/msup><\/mrow>1.92\\times10^{-6}<\/annotation><\/semantics><\/math> \u306b\u306a\u3063\u305f\u3002\u305f\u3060\u3057\u3001N<\/mi>=<\/mo>10000<\/mn><\/mrow>N=10000<\/annotation><\/semantics><\/math> \u306e 2.00<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>6<\/mn><\/mrow><\/msup><\/mrow>2.00\\times10^{-6}<\/annotation><\/semantics><\/math> \u3068 N<\/mi>=<\/mo>100000<\/mn><\/mrow>N=100000<\/annotation><\/semantics><\/math> \u306e\u5024\u306f\u307b\u307c\u540c\u3058\u3067\u3001\u6bce\u56de\u304d\u308c\u3044\u306b\u6e1b\u308b\u308f\u3051\u3067\u306f\u306a\u3044\u3002\u8868\u306e\u5272\u5408\u304c\u305f\u307e\u305f\u307e 0.5 \u306b\u3054\u304f\u8fd1\u304f\u306a\u308b\u306a\u3069\u3001\u4e71\u6570\u306e\u3086\u3089\u304e\u304c\u3042\u308b\u305f\u3081\u3067\u3042\u308b\u3002<\/li>\n<\/ul>\n\n\n\n
              \"\"<\/figure>\n\n\n\n
              \u56f33: <\/em>P<\/mi>P <\/annotation><\/semantics><\/math>\u3092 Bernoulli(0.5) \u306b\u56fa\u5b9a\u3057\u3001\u6a2a\u8ef8\u3092\u30e2\u30c7\u30eb\u306e\u30b3\u30a4\u30f3\u306e\u8868\u306e\u78ba\u7387 <\/em>q<\/mi>q<\/annotation><\/semantics><\/math>\u3001\u7e26\u8ef8\u3092 <\/em>D<\/mi>K<\/mi>L<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mo>Q<\/mi>)<\/mo><\/mrow>D_{KL}(P\u2016Q)<\/annotation><\/semantics><\/math>\uff08\u5358\u4f4d\u306f nat\uff09\u3068\u3057\u3066\u63cf\u3044\u305f\u56f3\u3002<\/em>q<\/mi>=<\/mo>0.5<\/mn><\/mrow>q=0.5<\/annotation><\/semantics><\/math> \u3067\u6700\u5c0f\u5024 0 \u306b\u306a\u308b\u3002<\/em><\/p>\n\n\n\n
              \u56f33\u3067\u306f\u3001\u6700\u5c0f\u5024\u304c q<\/mi>=<\/mo>0.5<\/mn><\/mrow>q=0.5<\/annotation><\/semantics><\/math> \u306e\u3068\u3053\u308d\u306b\u3042\u308b\u3002\u3053\u308c\u306f\u3001\u30b9\u30c6\u30c3\u30d73\u306e\u7b49\u53f7\u6761\u4ef6 P<\/mi>=<\/mo>Q<\/mi><\/mrow>P=Q<\/annotation><\/semantics><\/math> \u3068\u4e00\u81f4\u3059\u308b\u3002<\/p>\n\n\n\n
              \"\"<\/figure>\n\n\n\n
              \u56f34: <\/em>P<\/mi>P<\/annotation><\/semantics><\/math> \u3092 Bernoulli(0.5) \u306b\u56fa\u5b9a\u3057\u3001\u6a2a\u8ef8\u3092 <\/em>q<\/mi>q<\/annotation><\/semantics><\/math> \u3068\u3057\u3066\u3001\u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc <\/em>H<\/mi>(<\/mo>P<\/mi>,<\/mo>Q<\/mi>)<\/mo><\/mrow>H(P,Q)<\/annotation><\/semantics><\/math>\u3001\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc <\/em>H<\/mi>(<\/mo>P<\/mi>)<\/mo><\/mrow>H(P)<\/annotation><\/semantics><\/math>\uff08\u5b9a\u6570\uff09\u3001\u304a\u3088\u3073\u305d\u306e\u5dee <\/em>H<\/mi>(<\/mo>P<\/mi>,<\/mo>Q<\/mi>)<\/mo>\u2212<\/mo>H<\/mi>(<\/mo>P<\/mi>)<\/mo><\/mrow>H(P,Q)\u2212H(P)<\/annotation><\/semantics><\/math>\uff08KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306b\u4e00\u81f4\uff09\u3092\u91cd\u306d\u305f\u56f3\u3002<\/em><\/p>\n\n\n\n
              \u56f34\u3067\u306f\u3001\u4ea4\u5dee\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u3068\u30a8\u30f3\u30c8\u30ed\u30d4\u30fc\u306e\u5dee\u304c\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306e\u66f2\u7dda\u3068\u3057\u3066\u63cf\u304b\u308c\u3066\u3044\u308b\u3002\u5148\u306b\u793a\u3057\u305f\u6700\u5927\u8aa4\u5dee\u306e\u5c0f\u3055\u3055\u306f\u3001\u30b9\u30c6\u30c3\u30d72\u306e\u6052\u7b49\u5f0f\u3068\u6570\u5024\u7684\u306b\u6574\u5408\u3057\u3066\u3044\u308b\u3002<\/p>\n\n\n\n
              KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306e\u975e\u5bfe\u79f0\u6027\uff08P=Bernoulli(0.8)\uff09<\/p>\n\n\n\n
              \"\"<\/figure>\n\n\n\n
              \u56f35: <\/em>P<\/mi>P<\/annotation><\/semantics><\/math>\u3092 Bernoulli(0.8) \u306b\u56fa\u5b9a\u3057\u3001\u6a2a\u8ef8\u3092 <\/em>q<\/mi>q<\/annotation><\/semantics><\/math> \u3068\u3057\u3066\u3001<\/em>D<\/mi>K<\/mi>L<\/mi><\/mrow><\/msub>(<\/mo>P<\/mi>\u2016<\/mo>Q<\/mi>)<\/mo>\u3068<\/mtext>D<\/mi>K<\/mi>L<\/mi><\/mrow><\/msub>(<\/mo>Q<\/mi>\u2016<\/mo>P<\/mi>)<\/mo><\/mrow>D_{KL}(P\u2016Q) \u3068 D_{KL}(Q\u2016P)<\/annotation><\/semantics><\/math> \u306e\u66f2\u7dda\u3092\u4e26\u3079\u305f\u56f3\u30022\u3064\u304c\u4e00\u81f4\u3057\u306a\u3044\u3053\u3068\u304c\u5206\u304b\u308b\u3002<\/em><\/p>\n\n\n\n
              \u56f35\u3067\u306f\u3001P<\/mi>=<\/mo><\/mtext>Bernoulli<\/mi><\/mrow>(<\/mo>0.8<\/mn>)<\/mo><\/mrow>P=\\mathrm{Bernoulli}(0.8)<\/annotation><\/semantics><\/math> \u306b\u5bfe\u3059\u308b2\u3064\u306e\u5411\u304d\u306e\u66f2\u7dda\u304c\u4e00\u81f4\u3057\u3066\u3044\u306a\u3044\u3002q<\/mi>q<\/annotation><\/semantics><\/math> \u3092\u52d5\u304b\u3057\u305f\u7bc4\u56f2\u3067\u3001\u30b9\u30c6\u30c3\u30d74\u306e\u975e\u5bfe\u79f0\u6027\u3092\u78ba\u8a8d\u3067\u304d\u308b\u3002<\/p>\n\n\n\n
              \u6295\u3052\u308b\u56de\u6570 N<\/mi>N<\/annotation><\/semantics><\/math> \u3068\u7d4c\u9a13\u5206\u5e03\u306e KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9<\/p>\n\n\n\n
              \"\"<\/figure>\n\n\n\n
              \u56f36: \u30b3\u30a4\u30f3\u3092 N \u56de\uff0810 \u56de\u304b\u3089 100000 \u56de\uff09\u6295\u3052\u3066\u5f97\u305f\u7d4c\u9a13\u5206\u5e03\u3068\u3001\u771f\u306e\u30b3\u30a4\u30f3 Bernoulli(0.5) \u3068\u306e KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u3092\u3001\u4e21\u5bfe\u6570\u8ef8\u3067\u63cf\u3044\u305f\u56f3\u3002\u53c2\u8003\u3068\u3057\u3066 1\/(2N) \u306e\u66f2\u7dda\u3092\u7834\u7dda\u3067\u91cd\u306d\u3066\u3044\u308b\u3002<\/em><\/p>\n\n\n\n
              \u56f36\u3067\u306f\u3001N<\/mi>N<\/annotation><\/semantics><\/math> \u3092\u5897\u3084\u3059\u3068\u7d4c\u9a13\u5206\u5e03\u304c\u771f\u306e\u30b3\u30a4\u30f3\u306b\u8fd1\u3065\u304d\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u304c\u5c0f\u3055\u304f\u306a\u3063\u3066\u3044\u308b\u3002\u3053\u308c\u306f\u3001\u30b9\u30c6\u30c3\u30d73\u306e\u7b49\u53f7\u6761\u4ef6 P<\/mi>=<\/mo>Q<\/mi><\/mrow>P=Q<\/annotation><\/semantics><\/math> \u306b\u8fd1\u3065\u304f\u3053\u3068\u306b\u5bfe\u5fdc\u3059\u308b\u3002<\/p>\n\n\n\n
              \u7834\u7dda\u306f\u53c2\u8003\u3068\u3057\u3066\u63cf\u3044\u305f 1<\/mn>\/<\/mi>(<\/mo>2<\/mn>N<\/mi>)<\/mo><\/mrow>1\/(2N)<\/annotation><\/semantics><\/math> \u306e\u66f2\u7dda\u3067\u3042\u308b\u3002\u3053\u306e\u4fc2\u6570\u306f\u3001\u76ee\u5b89\u306e\u66f2\u7dda\u3067\u3042\u308a\u6e1b\u5c11\u306e\u69d8\u5b50\u3092\u7406\u8ad6\u7dda\u3068\u3057\u3066\u793a\u3059\u3082\u306e\u3067\u306f\u306a\u3044\u3002\u5b9f\u969b\u3001N<\/mi>=<\/mo>10000<\/mn><\/mrow>N=10000<\/annotation><\/semantics><\/math> \u3067\u306f 1<\/mn>\/<\/mi>(<\/mo>2<\/mn>N<\/mi>)<\/mo>=<\/mo>5<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>5<\/mn><\/mrow><\/msup><\/mrow>1\/(2N)=5\\times10^{-5}<\/annotation><\/semantics><\/math> \u3060\u304c\u3001\u5b9f\u6e2c\u306f 2.00<\/mn>\u00d7<\/mo>10<\/mn>\u2212<\/mo>6<\/mn><\/mrow><\/msup><\/mrow>2.00\\times10^{-6}<\/annotation><\/semantics><\/math> \u3060\u3063\u305f\u3002<\/p>\n\n\n\n
              \u78ba\u304b\u3081\u3089\u308c\u305f\u306e\u306f\u3001N<\/mi>N<\/annotation><\/semantics><\/math> \u304c\u5927\u304d\u3044\u307b\u3069\u5024\u304c\u304a\u304a\u3080\u306d\u5c0f\u3055\u304f\u306a\u308b\u3068\u3044\u3046\u50be\u5411\u307e\u3067\u3067\u3042\u308b\u3002\u3057\u304b\u30821\u56de\u306e\u8a66\u884c\u306e\u7d50\u679c\u306b\u3059\u304e\u306a\u3044\u306e\u3067\u3001\u7d30\u304b\u3044\u6e1b\u308a\u65b9\u307e\u3067\u306f\u3053\u306e\u5b9f\u9a13\u304b\u3089\u5224\u65ad\u3059\u308b\u3053\u3068\u306f\u3067\u304d\u306a\u3044\u3002<\/p>\n\n\n\n
              \u6ce8\u610f\u3068\u3088\u304f\u3042\u308b\u8aa4\u89e3<\/h2>\n\n\n\n
              \u3053\u306e\u5b9a\u7fa9\u304c\u6210\u308a\u7acb\u305f\u306a\u304f\u306a\u308b\u306e\u306f\u3001p<\/mi>(<\/mo>x<\/mi>)<\/mo>><\/mo>0<\/mn><\/mrow>p(x)>0<\/annotation><\/semantics><\/math> \u306a\u306e\u306b q<\/mi>(<\/mo>x<\/mi>)<\/mo>=<\/mo>0<\/mn><\/mrow>q(x)=0<\/annotation><\/semantics><\/math> \u3068\u306a\u308b\u5834\u5408\u3067\u3042\u308b\u3002\u4f8b\u3048\u3070\u30b3\u30a4\u30f3\u3067 q<\/mi>=<\/mo>0<\/mn><\/mrow>q=0<\/annotation><\/semantics><\/math> \u3068\u3059\u308b\u3068\u3001\u8868\u304c\u51fa\u308b P<\/mi>P<\/annotation><\/semantics><\/math> \u306b\u5bfe\u3057\u3066 log<\/mi>\u2061<\/mo><\/mspace><\/mrow>p<\/mi>q<\/mi><\/mfrac><\/mrow>\\log\\frac pq<\/annotation><\/semantics><\/math> \u304c\u767a\u6563\u3057\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306f\u7121\u9650\u5927\u306b\u306a\u308b [5]\u3002<\/p>\n\n\n\n
              \u8aa4\u89e3\u306e1\u3064\u76ee\u306f\u3001KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u3092\u8ddd\u96e2\u3068\u898b\u306a\u3059\u3053\u3068\u3067\u3042\u308b\u3002\u8ddd\u96e2\u3068\u3057\u3066\u306e\u91cf\u306f\u3001\u5bfe\u79f0\u3067\u3042\u308b\u3053\u3068\u3068\u4e09\u89d2\u4e0d\u7b49\u5f0f\u304c\u6210\u308a\u7acb\u3064\u3053\u3068\u304c\u5fc5\u8981\u306b\u306a\u308b\u3002\u4e09\u89d2\u4e0d\u7b49\u5f0f\u3068\u306f\u30013\u70b9 A\u3001B\u3001C \u306b\u3064\u3044\u3066\u3001A\u304b\u3089C\u307e\u3067\u306e\u8ddd\u96e2\u304c\u3001A\u304b\u3089B\u307e\u3067\u3068B\u304b\u3089C\u307e\u3067\u306e\u8ddd\u96e2\u306e\u548c\u4ee5\u4e0b\u306b\u306a\u308b\u3068\u3044\u3046\u6027\u8cea\u3067\u3042\u308b\u3002KL\u30c0\u30a4\u30d0\u30fc\u30b8\u30a7\u30f3\u30b9\u306f\u975e\u5bfe\u79f0\u3067\u3001\u4e09\u89d2\u4e0d\u7b49\u5f0f\u3082\u6210\u308a\u7acb\u305f\u306a\u3044\u306e\u3067\u3001\u8ddd\u96e2\u3068\u3057\u3066\u306f\u6271\u308f\u308c\u306a\u3044 [6][7]\u3002\u3069\u3061\u3089\u306e\u5411\u304d\u3092\u4f7f\u3046\u304b\u306f\u3001\u76ee\u7684\u306b\u5408\u308f\u305b\u3066\u9078\u629e\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b\u3002<\/p>\n\n\n\n
              \u8aa4\u89e3\u306e2\u3064\u76ee\u306f\u3001\u5024\u304c\u5c0f\u3055\u3051\u308c\u3070\u5411\u304d\u306b\u3088\u3089\u305a\u4f3c\u3066\u3044\u308b\u3068\u307f\u306a\u3059\u3053\u3068\u3067\u3042\u308b\u3002\u5411\u304d\u306b\u3088\u3063\u3066\u5024\u304c\u5909\u308f\u308b\u305f\u3081\u3001\u3069\u3061\u3089\u306e\u5411\u304d\u3092\u6e2c\u3063\u305f\u306e\u304b\u3092\u660e\u793a\u3059\u308b\u5fc5\u8981\u304c\u3042\u308b [8]\u3002<\/p>\n\n\n\n
              \u307e\u3068\u3081<\/h2>\n\n\n\n