Execution Time and Resource Usage Prediction for Immunology Research Software across Private Clouds

Abstract

Immunology research increasingly relies on complex software applications that require significant computing power. Due to data privacy and regulatory requirements, these tools are often run in private cloud environments where resources are fixed and limited. Currently, researchers find it difficult to predict how long a tool will take or what hardware resources it will consume before starting a run. This uncertainty often leads to choosing the wrong machine, resulting in either wasted resources or system crashes due to insufficient memory. This dissertation proposes an integrated system that provides estimates of execution time, cost, and hardware usage for immunology software. The system is designed to be easy to use, appearing directly within the platform researchers already use. To handle the initial lack of data, a three-phase approach is used: it starts with a mathematical model based on a questionnaire and evolves into machine learning models as the system collects data from past runs. Besides predictions, the system includes a dashboard to analyze historical performance data. This work contributes to a more efficient management of computing infrastructure in immunology, allowing researchers to focus on their scientific work with more confidence and less technical complexity.