LinearBoost: Faster than XGBoost and LightGBM, outperforming them on F1 Score on seven famous benchmark datasets

The latest version of LinearBoost classifier is released!

In benchmarks on 7 well-known datasets (Breast Cancer Wisconsin, Heart Disease, Pima Indians Diabetes Database, Banknote Authentication, Haberman’s Survival, Loan Status Prediction, and PCMAC), LinearBoost achieved these results:

  • It outperformed XGBoost on F1 score on all of the seven datasets

  • It outperformed LightGBM on F1 score on five of seven datasets

  • It reduced the runtime by up to 98% compared to XGBoost and LightGBM

  • It achieved competitive F1 scores with CatBoost, while being much faster

LinearBoost is a customized boosted version of SEFR, a super-fast linear classifier. It considers all of the features simultaneously instead of picking them one by one (as in Decision Trees), and so makes a more robust decision making at each step.

This is a side project, and authors work on it in their spare time. However, it can be a starting point to utilize linear classifiers in boosting to get efficiency and accuracy. The authors are happy to get your feedback!

1 Like