返回首页

对比阅读

对比阅读:RFE Explained: The "Survivor" Method for Pruning ML Features — Why More Data Isn't Always Better 与 机器学习删特征的"淘汰赛"方法 — 用一个心脏病案例讲清为什么数据不是越多越好

AEN
recursive feature eliminationmachine learningfeature engineering·

RFE Explained: The "Survivor" Method for Pruning ML Features — Why More Data Isn't Always Better

This Juejin article covers a foundational machine learning technique: Recursive Feature Elimination (RFE). The author uses a "company performance cut" analogy — train on all features first, let the model score each one's performance, drop the worst one, retrain, repeat, then select the feature subset with the highest cross-validation score from any round.

Running through a heart disease prediction case, out of 30 features, only 8–10 survive — and the model actually scores higher than when using all features. What the article doesn't mention: this "less is more" reasoning has shown up repeatedly in LLM (large language model, i.e. AI that can understand and generate text) training over the past two years. OpenAI, Anthropic, and Alibaba all use "quality filtering" to prune training data — the logic is essentially the same as this RFE piece.

What This Is

Recursive Feature Elimination (RFE) follows a four-step loop: train a base model (e.g. random forest or logistic regression) on all features → have the model score each feature's importance → drop the least important ones → retrain, score, drop, until only one feature remains. Each round runs cross-validation, and the final selection isn't "whatever's left" — it's the feature combination from the historical round that scored highest, i.e. the "inflection point."

At the code level, the author wraps a class with three core methods: _evaluate_subset uses cross-validation for scoring (avoiding the "overfitting" trap where a model only performs well on training data and collapses on new data), _extract_importance pulls feature importance based on model type (tree models use feature_importances_, linear models use the absolute value of coef_), and _locate_optimum picks the highest score from the historical log. Three output interfaces come bundled: select(), summary_table(), and plot_curve(). The plotted curve typically rises then falls — the peak is the optimal feature count.

Industry View

The technical depth here isn't high — sklearn (Python's most mainstream machine learning library) ships with its own RFE implementation, and the author mainly rebuilds the wheel while writing it in an accessible way. Precisely because of that, two camps coexist in the comments.

Supporters argue: this example is perfect for explaining the value of "Feature Engineering" (manually selecting and processing input data) to non-algorithm roles. Colleagues doing risk control, user segmentation, or sales forecasting can directly transplant the thinking. Critics counter: RFE has long been replaced in industry by automated feature selection (e.g. recursive methods based on SHAP values — a tool that explains each feature's contribution to predictions), and the author's use of a cleanly structured heart disease dataset is unrealistic — on real business data with thousands of columns and missing values everywhere, this method basically can't run.

One more controversy: the article ends with a cloud drive link for downloading the code. That's unusual outside Juejin and led some to wonder whether it's a soft-ad traffic play.

Impact on Regular People

For enterprise IT: If your company is rolling out a machine learning project, remember that "more data means a better model" is an illusion. Spending two weeks on feature selection early on often saves more trouble than swapping in a pricier model later.

For individual careers: The judgment framework of "cut what's unimportant, keep what's critical" applies to any prioritization scenario — not every customer request, job skill, or meeting agenda deserves equal investment.

For the consumer market: No direct impact is visible yet. This article targets developers and produces no perceptible change for ordinary consumers.

来源: juejin.cn
BZH
递归特征消除机器学习特征工程·

机器学习删特征的"淘汰赛"方法 — 用一个心脏病案例讲清为什么数据不是越多越好

这篇掘金文章介绍的是机器学习里的一个基础技巧:递归特征消除法(Recursive Feature Elimination,简称 RFE)。作者用"公司末位淘汰赛"做类比——先把全部特征拉进来训练,让模型给每个特征打绩效分,砍掉最差的那个,再训练,再砍,循环往复,最后挑出交叉验证(Cross Validation)得分最高的那一轮剩下的特征。

用一个心脏病预测的案例跑下来,30 个特征里最终可能只留 8-10 个,模型得分反而比全特征时高。文章没提的是:这种"少即是多"的思路,过去两年在 LLM(大语言模型,即能理解和生成文字的 AI 模型)训练里反复出现——OpenAI、Anthropic、阿里都在用"质量过滤"砍训练数据,逻辑和这篇 RFE 本质一致。

这是什么

递归特征消除(RFE)的核心流程是四步:用全部特征训练一个基模型(比如随机森林或逻辑回归)→ 让模型给每个特征打重要性分→ 删掉最不重要的若干个→ 重新训练,打分,删除,直到只剩一个特征。每一轮都做一次交叉验证,最后不选"剩下的",而是选历史得分最高那一轮的特征组合——也就是"拐点"。

代码层面作者封装了一个类,核心方法三个:_evaluate_subset 用交叉验证打分(避免模型只在训练集上表现好、换个数据就垮的"过拟合"陷阱)、_extract_importance 根据模型类型提取特征重要性(树模型看 feature_importances_,线性模型看 coef_ 绝对值)、_locate_optimum 从历史日志里挑最高分。配套还有 select()、summary_table()、plot_curve() 三个输出接口,画出来的曲线一般先升后降,峰值就是最优特征数。

行业怎么看

这篇文章的技术含量不算高——sklearn(Python 最主流的机器学习工具库)里自带 RFE 实现,作者主要是自己造轮子并写得很通俗。正因如此,评论区里两类声音并存。

支持方认为:这个例子很适合给非算法岗的人讲清楚"特征工程"(Feature Engineering,即人工挑选和加工输入数据)的价值,公司里做风控、做用户分群、做销量预测的同事都能直接迁移思路。批评方则指出:RFE 在工业界早就被自动化特征选择(比如基于 SHAP 值——一种解释每个特征对预测结果贡献大小的工具——的递归方法)取代,且作者用心脏病数据集这种结构干净的数据演示,在真实业务那种几千列、缺失值满天飞的数据上,这种方法基本跑不动。

还有一个争议:文章结尾附了一个网盘链接让读者下载代码,这点在掘金之外并不常见,也让一些人怀疑这是不是软广引流。

对普通人的影响

对企业 IT: 如果你们公司正在上马机器学习项目,记住"数据越多模型越好"是错觉。早期花两周时间做特征筛选,往往比后期换更贵的模型更省事。

对个人职场: "砍掉不重要的、留下关键的"这个判断框架,可以套用到任何需要做优先级排序的工作场景里——不是所有客户需求、所有岗位技能、所有会议议题都值得同等投入。

对消费市场: 目前还看不到直接影响。这篇文章面向的是开发者群体,对普通消费者不会产生任何可感知的变化。

来源: juejin.cn