Tree-based regression approaches are nonlinear models that work by partitioning the input space into cuboid regions. The prediction of the model is based on some aggregate statistic, such as the mean or the median of the training examples in the cuboid region that matches the unlabeled example. Owing to this, trained tree-based models are easily understood with no machine learning background leading to their wide deployment in industrial applications. There are many variants of tree-based approaches, such as the CART, ID3 and C4.5.
In this article, we focus on the CART-based regression approach.