{"id":7150,"date":"2025-09-17T10:46:49","date_gmt":"2025-09-17T01:46:49","guid":{"rendered":"https:\/\/blog.since2020.jp\/?p=7150"},"modified":"2025-09-17T10:50:54","modified_gmt":"2025-09-17T01:50:54","slug":"multiple_regression_analysis","status":"publish","type":"post","link":"https:\/\/since2020.jp\/media\/multiple_regression_analysis\/","title":{"rendered":"\u56de\u5e30\u5206\u6790\u3092\u6975\u3081\u308b(1\/4)\uff1a\u91cd\u56de\u5e30\u5206\u6790"},"content":{"rendered":"\n

\u56de\u5e30\u5206\u6790\u306f\u7d71\u8a08\u5b66\u306e\u4e2d\u3067\u3082\u6700\u3082\u57fa\u672c\u7684\u3067\u91cd\u8981\u306a\u624b\u6cd5\u306e\u4e00\u3064\u3067\u3059\u3002\u672c\u30b7\u30ea\u30fc\u30ba\u3067\u306f\u3001\u56de\u5e30\u5206\u6790\u306e\u7406\u8ad6\u304b\u3089\u5b9f\u88c5\u307e\u3067\u30014\u56de\u306b\u308f\u305f\u3063\u3066\u8a73\u3057\u304f\u89e3\u8aac\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u7b2c1\u56de\u76ee\u3068\u306a\u308b\u4eca\u56de\u306f\u3001\u91cd\u56de\u5e30\u5206\u6790\u306e\u57fa\u790e\u304b\u3089\u6b63\u5247\u5316\u624b\u6cd5\u307e\u3067\u5e45\u5e83\u304f\u53d6\u308a\u4e0a\u3052\u307e\u3059\u3002<\/p>\n\n\n

\u91cd\u56de\u5e30\u5206\u6790\u3068\u306f<\/h2>\n

\u91cd\u56de\u5e30\u5206\u6790\u306f\u3001\u8907\u6570\u306e\u8aac\u660e\u5909\u6570\u3092\u7528\u3044\u3066\u76ee\u7684\u5909\u6570\u3092\u4e88\u6e2c\u30fb\u8aac\u660e\u3059\u308b\u7d71\u8a08\u624b\u6cd5\u3067\u3059\u3002\u4e00\u822c\u7684\u306a\u91cd\u56de\u5e30\u30e2\u30c7\u30eb\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u8868\u73fe\u3055\u308c\u307e\u3059\uff1a<\/p>\r\n

\\[y = \\beta_0 + \\beta_1 x_1 + \\beta_2 x_2 + \\cdots + \\beta_p x_p + \\varepsilon\\]<\/p>\r\n

\u3053\u3053\u3067\u3001\\(y\\)\u306f\u76ee\u7684\u5909\u6570\u3001\\(x_1, x_2, \\ldots, x_p\\)\u306f\u8aac\u660e\u5909\u6570\u3001\\(\\beta_0, \\beta_1, \\ldots, \\beta_p\\)\u306f\u56de\u5e30\u4fc2\u6570\u3001\\(\\varepsilon\\)\u306f\u8aa4\u5dee\u9805\u3067\u3059\u3002<\/p>\n\n

\u6700\u5c0f\uff12\u4e57\u6cd5<\/h2>\n

\u56de\u5e30\u4fc2\u6570\u306e\u63a8\u5b9a\u306b\u306f\u6700\u5c0f\uff12\u4e57\u6cd5<\/strong>\u304c\u4e00\u822c\u7684\u306b\u7528\u3044\u3089\u308c\u307e\u3059\u3002\u3053\u308c\u306f\u6b8b\u5dee\u5e73\u65b9\u548c\u3092\u6700\u5c0f\u5316\u3059\u308b\u3053\u3068\u3067\u4fc2\u6570\u3092\u6c42\u3081\u308b\u624b\u6cd5\u3067\u3059\u3002<\/p>\r\n

\u6b8b\u5dee\u5e73\u65b9\u548c\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u5b9a\u7fa9\u3055\u308c\u307e\u3059\uff1a<\/p>\r\n

\\[RSS = \\sum_{i=1}^{n} (y_i – \\hat{y}_i)^2 = \\sum_{i=1}^{n} (y_i – \\beta_0 – \\beta_1 x_{i1} – \\cdots – \\beta_p x_{ip})^2\\]<\/p>\r\n

\u884c\u5217\u8868\u8a18\u3092\u7528\u3044\u308b\u3068\u3001\u56de\u5e30\u4fc2\u6570\u306e\u6700\u5c0f\uff12\u4e57\u63a8\u5b9a\u91cf\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<\/p>\r\n

\\[\\hat{\\boldsymbol{\\beta}} = (\\boldsymbol{X}^T\\boldsymbol{X})^{-1}\\boldsymbol{X}^T\\boldsymbol{y}\\]<\/p>\n\n

\u30e2\u30c7\u30eb\u8a55\u4fa1\u6307\u6a19<\/h2>\n\u6c7a\u5b9a\u4fc2\u6570<\/b>\r\n

\u6c7a\u5b9a\u4fc2\u6570<\/strong>\uff08\\(R^2\\)\uff09\u306f\u3001\u30e2\u30c7\u30eb\u304c\u30c7\u30fc\u30bf\u306e\u5909\u52d5\u3092\u3069\u306e\u7a0b\u5ea6\u8aac\u660e\u3067\u304d\u3066\u3044\u308b\u304b\u3092\u8868\u3059\u6307\u6a19\u3067\u3059\uff1a<\/p>\r\n

\\[R^2 = 1 – \\frac{RSS}{TSS} = 1 – \\frac{\\sum_{i=1}^{n}(y_i – \\hat{y}_i)^2}{\\sum_{i=1}^{n}(y_i – \\bar{y})^2}\\]<\/p>\r\n\u81ea\u7531\u5ea6\u8abf\u6574\u6e08\u307f\u6c7a\u5b9a\u4fc2\u6570<\/b>\r\n

\u6c7a\u5b9a\u4fc2\u6570\u306f\u8aac\u660e\u5909\u6570\u3092\u5897\u3084\u3059\u3068\u5fc5\u305a\u5897\u52a0\u3059\u308b\u305f\u3081\u3001\u30e2\u30c7\u30eb\u6bd4\u8f03\u306b\u306f\u81ea\u7531\u5ea6\u8abf\u6574\u6e08\u307f\u6c7a\u5b9a\u4fc2\u6570<\/strong>\u304c\u7528\u3044\u3089\u308c\u307e\u3059\uff1a<\/p>\r\n

\\[R_{adj}^2 = 1 – \\frac{RSS\/(n-p-1)}{TSS\/(n-1)}\\]<\/p>\n\n

\u6b63\u5247\u5316<\/h2>\n

\u9ad8\u6b21\u5143\u30c7\u30fc\u30bf\u3084\u591a\u91cd\u5171\u7dda\u6027\u306e\u554f\u984c\u306b\u5bfe\u51e6\u3059\u308b\u305f\u3081\u3001\u6b63\u5247\u5316<\/strong>\u624b\u6cd5\u304c\u7528\u3044\u3089\u308c\u307e\u3059\u3002\u6b63\u5247\u5316\u306f\u3001\u56de\u5e30\u4fc2\u6570\u306b\u30da\u30ca\u30eb\u30c6\u30a3\u3092\u8ab2\u3059\u3053\u3068\u3067\u904e\u5b66\u7fd2\u3092\u9632\u304e\u307e\u3059\u3002<\/p>\r\nL2\u6b63\u5247\u5316\uff08\u30ea\u30c3\u30b8\u56de\u5e30\uff09<\/b>\r\n

L2\u6b63\u5247\u5316<\/strong>\uff08\u30ea\u30c3\u30b8\u56de\u5e30<\/strong>\uff09\u3067\u306f\u3001\u56de\u5e30\u4fc2\u6570\u306e\u4e8c\u4e57\u548c\u306b\u30da\u30ca\u30eb\u30c6\u30a3\u3092\u8ab2\u3057\u307e\u3059\uff1a<\/p>\r\n

$$\\min_{\\boldsymbol{\\beta}} |\\boldsymbol{y} – \\boldsymbol{X}\\boldsymbol{\\beta}|^2 + \\lambda |\\boldsymbol{\\beta}|_2^2$$<\/p>\r\nL1\u6b63\u5247\u5316\uff08Lasso\u56de\u5e30\uff09<\/b>\r\n

L1\u6b63\u5247\u5316<\/strong>\uff08Lasso\u56de\u5e30<\/strong>\uff09\u3067\u306f\u3001\u56de\u5e30\u4fc2\u6570\u306e\u7d76\u5bfe\u5024\u306e\u548c\u306b\u30da\u30ca\u30eb\u30c6\u30a3\u3092\u8ab2\u3057\u307e\u3059\uff1a<\/p>\r\n

$$\\min_{\\boldsymbol{\\beta}} |\\boldsymbol{y} – \\boldsymbol{X}\\boldsymbol{\\beta}|^2 + \\lambda |\\boldsymbol{\\beta}|_1$$<\/p>\r\n

Lasso\u56de\u5e30\u306f\u5909\u6570\u9078\u629e\u6a5f\u80fd\u3092\u6301\u3061\u3001\u4e00\u90e8\u306e\u56de\u5e30\u4fc2\u6570\u3092\u6b63\u78ba\u306b0\u306b\u3057\u307e\u3059\u3002<\/p>\r\nElastic Net<\/b>\r\n

Elastic Net<\/strong>\u306f\u3001L1\u6b63\u5247\u5316\u3068L2\u6b63\u5247\u5316\u3092\u7d44\u307f\u5408\u308f\u305b\u305f\u624b\u6cd5\u3067\u3059\uff1a<\/p>\r\n

$$\\min_{\\boldsymbol{\\beta}} |\\boldsymbol{y} – \\boldsymbol{X}\\boldsymbol{\\beta}|^2 + \\lambda [(1-\\alpha)|\\boldsymbol{\\beta}|_2^2 + \\alpha|\\boldsymbol{\\beta}|_1]$$<\/p>\r\n

<\/p>\n\n

R\u306b\u3088\u308b\u5b9f\u88c5\uff1a\u6b63\u5247\u5316\u30d1\u30b9<\/h2>\n

\u4ee5\u4e0b\u3067\u306f\u3001glmnet\u30d1\u30c3\u30b1\u30fc\u30b8\u3092\u7528\u3044\u3066\u6b63\u5247\u5316\u30d1\u30b9\u3092\u53ef\u8996\u5316\u3057\u307e\u3059\u3002\u6b63\u5247\u5316\u30d1\u30b9\u3068<\/span>\u306f\u3001\u6b63\u5247\u5316\u30d1\u30e9\u30e1\u30fc\u30bf$\\lambda$\u3092\u5909\u5316\u3055\u305b\u305f\u3068\u304d\u306e\u56de\u5e30\u4fc2\u6570\u306e\u5909\u5316\u3092\u53ef\u8996\u5316\u3057\u305f\u3082\u306e\u3067\u3059\u3002\u3053\u308c\u306b\u3088\u308a\u3001\u3069\u306e\u5909\u6570\u304c\u3069\u306e\u7a0b\u5ea6\u306e\u6b63\u5247\u5316\u3067\u9664\u53bb\u3055\u308c\u308b\u304b\u3092\u7406\u89e3\u3067\u304d\u307e\u3059\u3002\u306a\u304a\u3001\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3068\u3057\u3066\u3001Boston\u4f4f\u5b85\u4fa1\u683c\u30c7\u30fc\u30bf\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059\u3002<\/p>\r\n

\r\n
# \u5fc5\u8981\u306a\u30e9\u30a4\u30d6\u30e9\u30ea\u306e\u8aad\u307f\u8fbc\u307f\r\nlibrary(glmnet)\r\nlibrary(MASS)\r\n\r\n# Boston\u4f4f\u5b85\u4fa1\u683c\u30c7\u30fc\u30bf\u306e\u6e96\u5099\r\ndata(Boston)\r\nX <- as.matrix(Boston[, -14])  # medv\u3092\u9664\u304f\r\ny <- Boston$medv\r\nX_scaled <- scale(X)  # \u30c7\u30fc\u30bf\u306e\u6a19\u6e96\u5316\r\n\r\n# \u6b63\u5247\u5316\u30d1\u30b9\u306e\u8a08\u7b97\r\nridge_fit <- glmnet(X_scaled, y, alpha = 0, lambda = exp(seq(5, -5, length.out = 100)))\r\nlasso_fit <- glmnet(X_scaled, y, alpha = 1, lambda = exp(seq(2, -5, length.out = 100)))\r\nelastic_fit <- glmnet(X_scaled, y, alpha = 0.5, lambda = exp(seq(2, -5, length.out = 100)))\r\n\r\n# \u6b63\u5247\u5316\u30d1\u30b9\u306e\u53ef\u8996\u5316\uff08\u4f59\u767d\u3092\u8abf\u6574\uff09\r\npar(mfrow = c(2, 2), mar = c(4, 4, 5.5, 2), oma = c(1, 1, 1, 1))\r\n\r\n# 1. \u30ea\u30c3\u30b8\u56de\u5e30\u30d1\u30b9\r\nplot(ridge_fit, xvar = \"lambda\", label = TRUE, main = \"Ridge Regression Path\")\r\n\r\n# 2. Lasso\u56de\u5e30\u30d1\u30b9\r\nplot(lasso_fit, xvar = \"lambda\", label = TRUE, main = \"Lasso Regression Path\")\r\n\r\n# 3. Elastic Net \u30d1\u30b9\r\nplot(elastic_fit, xvar = \"lambda\", label = TRUE, main = \"Elastic Net Path (\u03b1=0.5)\")\r\n\r\n# 4. \u4ea4\u5dee\u691c\u8a3c\u7d50\u679c\uff08Lasso\uff09\r\ncv_lasso <- cv.glmnet(X_scaled, y, alpha = 1)\r\nplot(cv_lasso, main = \"Lasso Cross-Validation\")\r\nabline(v = log(cv_lasso$lambda.min), col = \"red\", lty = 2)\r\nabline(v = log(cv_lasso$lambda.1se), col = \"blue\", lty = 2)\r\n\r\n# \u7d50\u679c\u306e\u8868\u793a\r\ncat(\"\u6700\u9069lambda\u5024:\\\\n\")\r\ncat(\"Lasso lambda.min:\", cv_lasso$lambda.min, \"\\\\n\")\r\ncat(\"Lasso lambda.1se:\", cv_lasso$lambda.1se, \"\\\\n\")\r\n\r\n# \u6700\u9069\u30d1\u30e9\u30e1\u30fc\u30bf\u3067\u306e\u4fc2\u6570\r\ncoef_lasso <- as.vector(coef(cv_lasso, s = cv_lasso$lambda.min))\r\ncoef_names <- rownames(coef(cv_lasso, s = cv_lasso$lambda.min))\r\n\r\n# \u975e\u30bc\u30ed\u4fc2\u6570\u306e\u62bd\u51fa\r\nnonzero_idx <- which(coef_lasso != 0)\r\nnonzero_coef <- data.frame(\r\n  Variable = coef_names[nonzero_idx],\r\n  Coefficient = coef_lasso[nonzero_idx]\r\n)\r\n\r\ncat(\"\\\\nLasso\u306e\u975e\u30bc\u30ed\u4fc2\u6570:\\\\n\")\r\nprint(nonzero_coef)<\/code><\/pre>\r\n<\/div>\r\n
\u51fa\u529b\u7d50\u679c<\/pre>\r\n
\r\n
\u6700\u9069lambda\u5024:\r\nLasso lambda.min: 0.02800535\r\nLasso lambda.1se: 0.4158705\r\n\r\nLasso\u306e\u975e\u30bc\u30ed\u4fc2\u6570:\r\n      Variable Coefficient\r\n1  (Intercept)  22.5328063\r\n2         crim  -0.8459305\r\n3           zn   0.9665200\r\n4         chas   0.6820217\r\n5          nox  -1.8895761\r\n6           rm   2.7169775\r\n7          dis  -2.9396168\r\n8          rad   2.2002069\r\n9          tax  -1.6561804\r\n10     ptratio  -2.0133748\r\n11       black   0.8240161\r\n12       lstat  -3.7312035<\/code><\/pre>\r\n<\/div>\r\n
\"\"<\/p>\n\n
\u6b63\u5247\u5316\u30d1\u30b9\u306e\u89e3\u91c8<\/h2>\n
\u4f5c\u6210\u3055\u308c\u305f4\u3064\u306e\u30d7\u30ed\u30c3\u30c8\u304b\u3089\u4ee5\u4e0b\u306e\u3053\u3068\u304c\u5206\u304b\u308a\u307e\u3059\uff1a<\/p>\r\n
    \r\n\t
  1. Ridge Regression Path<\/strong>: \u30ea\u30c3\u30b8\u56de\u5e30\u3067\u306f\u4fc2\u6570\u306f0\u306b\u8fd1\u3065\u304d\u307e\u3059\u304c\u3001\u5b8c\u5168\u306b\u306f0\u306b\u306a\u308a\u307e\u305b\u3093<\/li>\r\n\t
  2. Lasso Regression Path<\/strong>: Lasso\u56de\u5e30\u3067\u306f\u591a\u304f\u306e\u4fc2\u6570\u304c\u6b63\u78ba\u306b0\u306b\u306a\u308a\u3001\u5909\u6570\u9078\u629e\u304c\u884c\u308f\u308c\u307e\u3059<\/li>\r\n\t
  3. Elastic Net Path<\/strong>: L1\u3068L2\u306e\u4e2d\u9593\u7684\u306a\u6027\u8cea\u3092\u793a\u3057\u3001\u6bb5\u968e\u7684\u306b\u5909\u6570\u304c\u9664\u53bb\u3055\u308c\u307e\u3059<\/li>\r\n\t
  4. \u4ea4\u5dee\u691c\u8a3c\u7d50\u679c<\/strong>: \u6700\u9069\u306a\u03bb\u5024\u3092\u5ba2\u89b3\u7684\u306b\u9078\u629e\u3067\u304d\u307e\u3059<\/li>\r\n<\/ol>\r\n
    <\/p>\n\n
    \u307e\u3068\u3081<\/h2>\n
    \u672c\u8a18\u4e8b\u3067\u306f\u3001\u91cd\u56de\u5e30\u5206\u6790\u306e\u57fa\u790e\u304b\u3089\u6b63\u5247\u5316\u624b\u6cd5\u307e\u3067\u89e3\u8aac\u3057\u3001\u5b9f\u969b\u306eR\u30b3\u30fc\u30c9\u3067\u6b63\u5247\u5316\u30d1\u30b9\u3092\u53ef\u8996\u5316\u3057\u307e\u3057\u305f\u3002\u91cd\u8981\u306a\u30dd\u30a4\u30f3\u30c8\u306f\u4ee5\u4e0b\u306e\u901a\u308a\u3067\u3059\u3002<\/p>\r\n
      \r\n\t
    1. \u6700\u5c0f\uff12\u4e57\u6cd5<\/strong>\u306f\u56de\u5e30\u4fc2\u6570\u63a8\u5b9a\u306e\u57fa\u672c\u624b\u6cd5<\/li>\r\n\t
    2. \u6c7a\u5b9a\u4fc2\u6570<\/strong>\u3067\u30e2\u30c7\u30eb\u6027\u80fd\u3092\u8a55\u4fa1<\/li>\r\n\t
    3. \u6b63\u5247\u5316<\/strong>\u306b\u3088\u308a\u904e\u5b66\u7fd2\u3068\u591a\u91cd\u5171\u7dda\u6027\u306e\u554f\u984c\u3092\u89e3\u6c7a<\/li>\r\n\t
    4. L1\u6b63\u5247\u5316<\/strong>\u306f\u5909\u6570\u9078\u629e\u3001L2\u6b63\u5247\u5316<\/strong>\u306f\u4fc2\u6570\u306e\u5b89\u5b9a\u5316\u306b\u6709\u52b9<\/li>\r\n\t
    5. \u6b63\u5247\u5316\u30d1\u30b9<\/strong>\u306b\u3088\u308a\u6700\u9069\u306a\u30d1\u30e9\u30e1\u30fc\u30bf\u3092\u8996\u899a\u7684\u306b\u7406\u89e3<\/li>\r\n<\/ol>\r\n
      \u6b21\u56de\u306f\u56de\u5e30\u8a3a\u65ad\u6cd5\u306b\u3064\u3044\u3066\u8a73\u3057\u304f\u89e3\u8aac\u3057\u3001\u30e2\u30c7\u30eb\u306e\u59a5\u5f53\u6027\u3092\u691c\u8a3c\u3059\u308b\u624b\u6cd5\u3092\u5b66\u3073\u307e\u3059\u3002<\/p>\r\n
      <\/p>","protected":false},"excerpt":{"rendered":"
      \u56de\u5e30\u5206\u6790\u306f\u7d71\u8a08\u5b66\u306e\u4e2d\u3067\u3082\u6700\u3082\u57fa\u672c\u7684\u3067\u91cd\u8981\u306a\u624b\u6cd5\u306e\u4e00\u3064\u3067\u3059\u3002\u672c\u30b7\u30ea\u30fc\u30ba\u3067\u306f\u3001\u56de\u5e30\u5206\u6790\u306e\u7406\u8ad6\u304b\u3089\u5b9f\u88c5\u307e\u3067\u30014\u56de\u306b\u308f\u305f\u3063\u3066\u8a73\u3057\u304f\u89e3\u8aac\u3057\u3066\u3044\u304d\u307e\u3059\u3002\u7b2c1\u56de\u76ee\u3068\u306a\u308b\u4eca\u56de\u306f\u3001\u91cd\u56de\u5e30\u5206\u6790\u306e\u57fa\u790e\u304b\u3089\u6b63\u5247\u5316\u624b\u6cd5\u307e\u3067\u5e45\u5e83\u304f\u53d6\u308a\u4e0a\u3052\u307e\u3059\u3002 \u91cd […]<\/p>\n","protected":false},"author":89,"featured_media":7137,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"content-type":"","swell_btn_cv_data":"","footnotes":"","_wp_rev_ctl_limit":""},"categories":[1246],"tags":[894,484,57],"class_list":["post-7150","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-infrastructure","tag-r","tag-484","tag-57"],"_links":{"self":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/7150","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/users\/89"}],"replies":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/comments?post=7150"}],"version-history":[{"count":0,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/posts\/7150\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media\/7137"}],"wp:attachment":[{"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/media?parent=7150"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/categories?post=7150"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/since2020.jp\/media\/wp-json\/wp\/v2\/tags?post=7150"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}