Chapter 6 Conclusion

Now after having a solid theoretical understanding of gradient boosting methods for regression and the implementation XGBoost as well as seeing the application of such models to two data sets one can wrap up this project by discussing the major pros and cons of this approach with this specific implementation in mind.

6.1 Pros

• Minimal pre-processing

• Flexible enough to detect complex non-linear patterns

• Handling of missing values

• Integrated feature selection

• Good generalization due to lots of regularization options

• Strong predictive power

6.2 Cons

• Not as explainable as for example a linear model

• Computationally demanding (especially the hyperparameter tuning)

• Preferably lots of observations

These pros and cons show that tree-based gradient boosting is a very powerful learning algorithm but still not suitable to any application. Due to its very robust nature and good handling of missing values and features of different scales it is one of the dominant algorithms in data mining alongside models like random forest. But in critical applications with just few observations such models should be treated with care as they are not that explainable. Still if one handles tabular data in order to perform a regression or classification task one should take the pros and cons that were mentioned above into account and in many cases a consideration of a tree-based gradient boosting model like XGBoost is advisable.

Thanks for sticking with it!