Tree-based classification approaches are nonlinear models that work by partitioning the input space into cuboid regions. The prediction of the model is based on the most dominant class represented by training examples in the cuboid region that matches the unlabeled example. Owing to this, trained tree-based models are easily understood with no machine learning background leading to their wide deployment in industrial applications. There are many variants of tree-based approaches, such as the CART, ID3 and C4.5.
In this article, we focus on the CART-based decision tree classifier. Then, we comment on the extensions in ID3 and C4.5 in later sections.