Climbing trees 1: what are decision trees?

Published: Jan 7, 2025 This is the first in a se­ries of posts about de­ci­sion trees in the con­text of ma­chine learn­ing. The goal here is to pro­vide a foun­da­tional un­der­stand­ing of de­ci­sion trees and to im­ple­ment them. Climb­ing trees se­ries De­ci­sion trees are not amaz­ing al­go­rithms by them­selves. They have lim­i­ta­tions that can re­sult in sub­op­ti­mal and even weird pre­dic­tions. And yet, they have be­come ex­tremely pop­u­lar. Some would even say they are the de facto go-to al­go­rithm for many ma­chine learn­ing do­mains. This is due to bag­ging and boost­ing, tech­niques that turned sub­par de­ci­sion trees into state-​of-​the-​art al­go­rithms. We’ll ex­plore them in the fu­ture. First, we’ll build an in­tu­ition for what are de­ci­sion trees and de­fine them math­e­mat­i­cally. Then, we’ll ex­plore how de­ci­sion trees are built. This will allow us to grasp their main char­ac­ter­is­tics, ad­van­tages and dis­ad­van­tages. I will try to in­tro­duce co ... Read full article.