Code summarization, creating readable descriptions from code snippets, is one of the popular research subfields in Machine Learning on Code. However, it is heavily dominated by research focusing on performance improvement. In real-world ML production systems, performance is only a fraction of decisions and criteria that must be considered. In this work, we presented the Meta-Learning Module (MLM), which uses surrogate models to track the performance of costly summarization algorithms. We can use MLM as a regression task, helping us monitor and improve the performance by enabling a more robust topic drift model selection process and creating a mixture of expert ensembles. It can also be used as a classification task and extended with the interpretability module, which supports us in understanding the potential model’s performance and the reason for it even before we run the costly summarization algorithm. Finally, MLM can support sudden data drift monitoring, which is the most challenging type of data drift.