Decision trees: Gini, entropy and greedy splits
How a decision tree picks each split greedily by Gini impurity or entropy, and why a tree grown until every leaf is pure overfits.
Part of Trees and ensembles.
Read next
- Random forests: bagging, bootstrap and decorrelated trees
Why averaging trees helps: bagging (bootstrap aggregating), out-of-bag error, and how random forests lower tree correlation ρ in ρσ² + (1−ρ)σ²/B.
Related
- Finish first
The greedy choice for interval scheduling: book the meeting that ends first to fit the most, why other greedy rules fail, and the stays-ahead proof.