{"id":9964,"date":"2026-10-07T10:16:54","date_gmt":"2026-10-07T01:16:54","guid":{"rendered":"https:\/\/since2020.jp\/media\/?p=9964"},"modified":"2026-10-07T10:16:54","modified_gmt":"2026-10-07T01:16:54","slug":"randomforest-feature-importance-pitfall","status":"publish","type":"post","link":"https:\/\/since2020.jp\/media\/randomforest-feature-importance-pitfall\/","title":{"rendered":"\u7279\u5fb4\u91cf\u91cd\u8981\u5ea6\u306f\u4fe1\u7528\u3057\u3066\u3044\u3044\u306e\u304b\uff1aRandomForest\u306f\u3001\u7121\u95a2\u4fc2\u306a\u30e9\u30f3\u30c0\u30e0\u5217\u3092\u300c\u6700\u91cd\u8981\u300d\u3068\u7b54\u3048\u308b"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">\u306f\u3058\u3081\u306b<\/h2>\n\n\n\n<p>RandomForest\u306b\u306f\u3001feature_importances_\u3068\u3044\u3046\u3001\u7279\u5fb4\u91cf\u306e\u91cd\u8981\u5ea6\u3092\u8fd4\u3059\u6a5f\u80fd\u304c\u3042\u308a\u307e\u3059\u3002\u300c\u3069\u306e\u5217\u304c\u4e88\u6e2c\u306b\u52b9\u3044\u3066\u3044\u308b\u304b\u300d\u3092\u624b\u8efd\u306b\u8abf\u3079\u3089\u308c\u308b\u305f\u3081\u3001\u3088\u304f\u4f7f\u308f\u308c\u307e\u3059\u3002\u3067\u306f\u3001\u3053\u306e\u91cd\u8981\u5ea6\u306f\u3001\u305d\u306e\u307e\u307e\u4fe1\u7528\u3057\u3066\u3082\u3044\u3044\u306e\u3067\u3057\u3087\u3046\u304b\u3002<\/p>\n\n\n\n<p>\u3053\u306e\u8a18\u4e8b\u3067\u306f\u3001\u4e88\u6e2c\u306b\u7121\u95a2\u4fc2\u306a\u30e9\u30f3\u30c0\u30e0\u5217\u3092\u6df7\u305c\u3066\u3001\u6b21\u306e\uff13\u70b9\u3092\u78ba\u304b\u3081\u307e\u3059\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6a19\u6e96\u306e\u91cd\u8981\u5ea6\u304c\u3001\u7121\u95a2\u4fc2\u306a\u5217\u3092\u6700\u91cd\u8981\u3068\u7b54\u3048\u308b\u3053\u3068<\/li>\n\n\n\n<li>\u305d\u306e\u539f\u56e0\u304c\u3001\u91cd\u8981\u5ea6\u306e\u8a08\u7b97\u65b9\u6cd5\u306e\u3069\u3053\u306b\u3042\u308b\u306e\u304b<\/li>\n\n\n\n<li>\u4fe1\u7528\u3067\u304d\u308b\u91cd\u8981\u5ea6\u306e\u8abf\u3079\u65b9<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">\u5b9f\u9a13\u306e\u8a2d\u5b9a<\/h2>\n\n\n\n<p>1000\u884c\u306e\u30c7\u30fc\u30bf\u3092\u4f5c\u308a\u307e\u3059\u3002\u30e9\u30d9\u30eb\u304c\uff11\u306b\u306a\u308b\u78ba\u7387\u306f\u3001\u6b21\u306e\uff12\u3064\u306e\u5217\u304c\u5927\u304d\u3044\u307b\u3069\u9ad8\u304f\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>binary:<\/strong>\uff10\u304b1\u306e\u4e8c\u5024\u306e\u5217(\u52b9\u304f)<\/li>\n\n\n\n<li><strong>level5:<\/strong>0~4\u306e5\u6bb5\u968e\u306e\u5217(\u52b9\u304f\uff09<\/li>\n<\/ul>\n\n\n\n<p>\u307e\u305f\u3001\u30e9\u30d9\u30eb\u3068\u7121\u95a2\u4fc2\u306a\u5217\u3068\u3057\u3066\u3001random_id\u3092\u52a0\u3048\u307e\u3059\u3002 0~1\u306e\u4e71\u6570\u3067\u3001\u5168\u884c\u3067\u5024\u304c\u9055\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import numpy as np\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.inspection import permutation_importance\nfrom sklearn.model_selection import train_test_split\n\nrng = np.random.default_rng(0)\nn = 1000\nb = rng.integers(0, 2, n)                 # \u52b9\u304f\u4e8c\u5024\u306e\u5217\nc = rng.integers(0, 5, n)                 # \u52b9\u304f5\u6bb5\u968e\u306e\u5217\np = 0.15 + 0.3 * b + 0.08 * c             # \u30e9\u30d9\u30eb\u304c1\u306b\u306a\u308b\u78ba\u7387\ny = (rng.uniform(size=n) &lt; p).astype(int)\nrnd = rng.uniform(size=n)                 # \u7121\u95a2\u4fc2\u306aID\u98a8\u306e\u5217\n\nX = np.column_stack(&#091;b, c, rnd])\nXtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.3, random_state=0)\n\nrf = RandomForestClassifier(n_estimators=300, random_state=0).fit(Xtr, ytr)\n\nprint(rf.feature_importances_)            # \u6a19\u6e96\u306e\u91cd\u8981\u5ea6\npi = permutation_importance(rf, Xte, yte, n_repeats=20, random_state=0)\nprint(pi.importances_mean)                # permutation importance<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">\u7d50\u679c\uff1a\u7121\u95a2\u4fc2\u306a\u5217\u304c\u3001\u91cd\u8981\u5ea6\u306e8\u5272\u3092\u5360\u3081\u308b<\/h2>\n\n\n\n<p>\u30c7\u30fc\u30bf\u306e\u4e71\u6570\u3092\u5909\u3048\u3066\u300120\u56de\u7e70\u308a\u8fd4\u3057\u305f\u5e73\u5747\u3067\u3059\u3002<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><tbody><tr><td>\u5217<\/td><td>\u5b9f\u969b\u306e\u95a2\u4fc2<\/td><td>\u6a19\u6e96\u306e\u91cd\u8981\u5ea6<\/td><td>permutation importance<\/td><\/tr><tr><td>binary<\/td><td>\u52b9\u304f<\/td><td>0.092<\/td><td>+0.034<\/td><\/tr><tr><td>level5<\/td><td>\u52b9\u304f<\/td><td>0.094<\/td><td>+0.012<\/td><\/tr><tr><td>random_id<\/td><td>\u7121\u95a2\u4fc2<\/td><td>0.814<\/td><td>-0.017<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>\u6a19\u6e96\u306e\u91cd\u8981\u5ea6\u3067\u306f\u3001\u7121\u95a2\u4fc2\u306arandom_id\u304c\u91cd\u8981\u5ea6\u306e\u7d048\u5272\u3092\u5360\u3081\u300120\u56de\u3059\u3079\u3066\u30671\u4f4d\u3067\u3057\u305f\u3002\u52b9\u3044\u3066\u3044\u308b2\u3064\u306e\u5217\u306f\u3001\u5408\u308f\u305b\u3066\u30822\u5272\u306b\u5c4a\u304d\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<p>\u4e00\u65b9\u3001permutation importance\u3067\u306f\u3001random_id\u306f20\u56de\u306e\u3046\u30611\u4f4d\u306b\u306a\u3063\u305f\u56de\u304c\u306a\u304f\u3001\u5e73\u5747\u306f\u8ca0\u306e\u5024\u306b\u306a\u308a\u307e\u3057\u305f\u3002\u52b9\u304f\u5217\u306e\u65b9\u304c\u3001\u9ad8\u3044\u5024\u306b\u306a\u3063\u3066\u3044\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u539f\u56e0\uff1a\u8a13\u7df4\u30c7\u30fc\u30bf\u3067\u8a08\u7b97\u3057\u3066\u3044\u308b\u304b\u3089<\/h2>\n\n\n\n<p>\u6a19\u6e96\u306e\u91cd\u8981\u5ea6\u306f\u3001\u6728\u304c\u5206\u5272\u3059\u308b\u305f\u3073\u306b\u6e1b\u3063\u305f\u300c\u4e0d\u7d14\u5ea6\u300d\uff08\u30e9\u30d9\u30eb\u306e\u3070\u3089\u3064\u304d\uff09\u3092\u3001\u8a13\u7df4\u30c7\u30fc\u30bf\u3067\u5408\u8a08\u3057\u305f\u3082\u306e\u3067\u3059\u3002<\/p>\n\n\n\n<p>\u5024\u306e\u7a2e\u985e\u304c\u591a\u3044\u5217\u306f\u3001\u5206\u5272\u306e\u5019\u88dc\u304c\u591a\u304f\u306a\u308a\u307e\u3059\u3002random_id\u306f\u5168\u884c\u3067\u5024\u304c\u9055\u3046\u305f\u3081\u3001\u305f\u307e\u305f\u307e\u30e9\u30d9\u30eb\u3092\u3046\u307e\u304f\u5207\u308a\u5206\u3051\u3089\u308c\u308b\u5206\u5272\u70b9\u3092\u3001\u3044\u304f\u3089\u3067\u3082\u898b\u3064\u3051\u3089\u308c\u307e\u3059\u3002\u6728\u306f\u6df1\u304f\u6210\u9577\u3059\u308b\u306e\u3067\u3001\u3053\u306e\u5076\u7136\u306e\u5206\u5272\u3082\u3001\u305d\u306e\u307e\u307e\u5b66\u7fd2\u3057\u307e\u3059\u3002\u3053\u306e\u5b9f\u9a13\u3067\u306f\u3001\u8a13\u7df4\u30c7\u30fc\u30bf\u3067\u306e\u6b63\u89e3\u7387\u304c1.0\u306b\u306a\u308a\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<p>\u3064\u307e\u308a\u3001\u6a19\u6e96\u306e\u91cd\u8981\u5ea6\u306f\u3001\u300c\u8a13\u7df4\u30c7\u30fc\u30bf\u306b\u3069\u308c\u3060\u3051\u5f53\u3066\u306f\u307e\u3063\u305f\u304b\u300d\u3092\u6e2c\u3063\u3066\u3044\u307e\u3059\u3002\u4e88\u6e2c\u306b\u672c\u5f53\u306b\u52b9\u304f\u304b\u306f\u6e2c\u308c\u3066\u3044\u307e\u305b\u3093\u3002<\/p>\n\n\n\n<p>\u3053\u306e\u73fe\u8c61\u306f\u3001\u5024\u306e\u7a2e\u985e\u304c\u591a\u3044\u5217\u3068\u3001\u6df1\u304f\u6210\u9577\u3057\u305f\u6728\u3067\u304a\u304d\u3084\u3059\u3044\u3067\u3059\u3002min_samples_leaf\u3092\u5927\u304d\u304f\u3059\u308b\u306a\u3069\u3067\u8efd\u304f\u306a\u308b\u5834\u5408\u3082\u3042\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u5bfe\u51e6\u6cd5\uff1a\u30c6\u30b9\u30c8\u30c7\u30fc\u30bf\u3067\u3001\u30b7\u30e3\u30c3\u30d5\u30eb\u3057\u3066\u6e2c\u308b<\/h2>\n\n\n\n<p>permutation importance\u306f\u3001\u30c6\u30b9\u30c8\u30c7\u30fc\u30bf\u306e\u3042\u308b\u5217\u306e\u5024\u3092\u30b7\u30e3\u30c3\u30d5\u30eb\u3057\u3066\u3001\u4e88\u6e2c\u306e\u7cbe\u5ea6\u304c\u3069\u308c\u3060\u3051\u4e0b\u304c\u308b\u304b\u3092\u6e2c\u308a\u307e\u3059\u3002\u7121\u95a2\u4fc2\u306a\u5217\u306f\u3001\u30b7\u30e3\u30c3\u30d5\u30eb\u3057\u3066\u3082\u7cbe\u5ea6\u304c\u5909\u308f\u3089\u306a\u3044\u306e\u3067\u3001\u91cd\u8981\u5ea6\u304c\u307b\u307c\uff10\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\n\n\n\n<p>\u3082\u3046\u4e00\u3064\u3001\u7121\u95a2\u4fc2\u306a\u4e71\u6570\u306e\u5217\u3092\uff11\u672c\u6df7\u305c\u3066\u304a\u304f\u65b9\u6cd5\u304c\u3042\u308a\u307e\u3059\u3002\u305d\u306e\u5217\u306e\u91cd\u8981\u5ea6\u3088\u308a\u4f4e\u3044\u7279\u5fb4\u91cf\u306f\u3001\u4fe1\u7528\u3067\u304d\u307e\u305b\u3093\u3002\u4eca\u56de\u306e\u5b9f\u9a13\u3067\u306f\u3001permutation importance\u3067\u3001\u52b9\u304f2\u3064\u306e\u5217\u304crandom_id\u3088\u308a\u4e0a\u306b\u4e26\u3073\u307e\u3057\u305f\u3002<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">\u307e\u3068\u3081<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u6a19\u6e96\u306e\u91cd\u8981\u5ea6\u306f\u3001\u7121\u95a2\u4fc2\u306a\u30e9\u30f3\u30c0\u30e0\u5217\u3092\u3001\u91cd\u8981\u5ea6\u306e\u7d048\u5272\u300120\u56de\u3059\u3079\u3066\u3067\uff11\u4f4d\u3068\u7b54\u3048\u305f<\/li>\n\n\n\n<li>\u539f\u56e0\u306f\u3001\u8a13\u7df4\u30c7\u30fc\u30bf\u3067\u8a08\u7b97\u3057\u3066\u3044\u308b\u305f\u3081\u3002\u5024\u306e\u7a2e\u985e\u304c\u591a\u3044\u5217\u306f\u5076\u7136\u306e\u5206\u5272\u3067\u5f97\u3092\u3059\u308b<\/li>\n\n\n\n<li>permutation importance\u3067\u306f\u3001random_id\u304c1\u4f4d\u306b\u306a\u308b\u56de\u306f\u306a\u304b\u3063\u305f\u3002\u30c6\u30b9\u30c8\u30c7\u30fc\u30bf\u3067\u6e2c\u308b\u65b9\u304c\u3001\u52b9\u304f\u5217\u3092\u6b63\u3057\u304f\u4e26\u3079\u3089\u308c\u308b<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>\u306f\u3058\u3081\u306b RandomForest\u306b\u306f\u3001feature_importances_\u3068\u3044\u3046\u3001\u7279\u5fb4\u91cf\u306e\u91cd\u8981\u5ea6\u3092\u8fd4\u3059\u6a5f\u80fd\u304c\u3042\u308a\u307e\u3059\u3002\u300c\u3069\u306e\u5217\u304c\u4e88\u6e2c\u306b\u52b9\u3044\u3066\u3044\u308b\u304b\u300d\u3092\u624b\u8efd\u306b\u8abf\u3079\u3089\u308c\u308b\u305f\u3081\u3001\u3088\u304f\u4f7f\u308f\u308c\u307e\u3059\u3002\u3067\u306f\u3001\u3053\u306e\u91cd\u8981\u5ea6\u306f\u3001\u305d\u306e\u307e [&hellip;]<\/p>\n","protected":false},"author":99,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","swell_btn_cv_data":"","footnotes":"","_wp_rev_ctl_limit":""},"categories":[1249],"tags":[39,26,419],"class_list":["post-9964","post","type-post","status-publish","format-standard","hentry","category-knowledge","tag-39","tag-26","tag-419"],"_links":{"self":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/9964","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/users\/99"}],"replies":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/comments?post=9964"}],"version-history":[{"count":2,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/9964\/revisions"}],"predecessor-version":[{"id":10055,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/9964\/revisions\/10055"}],"wp:attachment":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media?parent=9964"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/categories?post=9964"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/tags?post=9964"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}