Updated: Nov 11, 2019
Online Resources to Learn About Machine Learning Model Stacking
White and Scientific Articles on Machine Learning Model Ensembling
Stacked Generalization, DH Wolpert, Neural Networks, 1992
Overview of Recommended Resources
Ensembling or stacking refers to procedures designed to increase the predictive performance of machine learning algorithms, by blending or combining the predictions of multiple models. There are multiple ensembling or stacking methods. Simple stacking methods include voting or averaging the predictions. More complex ensembling methods aim to build new machine learning models (i.e., logistic regressions, k-nearest neighbours, boosting trees, etc) using the predictions as features.
To learn about stacking of machine learning models, the Kaggle ensembling guide blog written by one of Kaggle top competitors is the best place to start. In the Kaggle ensembling guide blog, the author describes a variety of stacking methods, and how he and his team have used them to win Kaggle competitions. As a bonus, he provides a number of resources for further reading.
In the Kaggle Homsite quote conversion articles and in the Otto product classification articles in Kaggle (see above for links), the Kaggle winners describe how by creatively implementing model stacking, with 2 or even 3 layers of machine learning model stacking, the authors managed to climb to the top of the competition board.These articles will give you a good understanding of how these techniques can be used to improve the performance of your machine learning models, and about the advantages and limitations of model stacking.
In the white and scientific articles recommended at the top of this page, the authors describe what model stacking is, and the different ways in which they have implemented stacking in a variety of data. They discuss in detail their advantages over single models. The articles also discuss the limitations encountered when trying to use these complicated models in real business scenarios. Scientific articles are usually very rich in the mathematical formulations that explain the functionality of the process.
Disclaimer: Opinions are my own and I do not become financial compensation from any of the links included in this article.