We develop HTBoost (Hybrid Tree Boosting), a model designed to retain the strengths of gradient boosting machines while addressing some of their limitations. Hybrid trees may contain any mix of standard hard splits and more flexible soft splits, adopting a two-step procedure to escape the local minima in which soft splits are sometimes trapped. Additionally, the fitted values of each tree are enhanced by a nonlinear transformation. These modifications increase accuracy when approximating functions that are smooth with respect to at least some features. The efficiency gains come at the cost of much longer computing times, particularly for large samples. The performance of HTBoost, XGBoost, and LightGBM is compared in simulations and in a few well-known datasets, where the realized accuracy gains are in line with what predicted by summary measures of smoothness. Finally, we extend the automatic treatment of missing values to smooth and hybrid trees.
The paper is available here.