Automated machine learning (AutoML) could make radiomics more accessible by automating complex modeling steps and reducing the programming expertise required to develop predictive models. Yet, the growing number of available frameworks raises a practical question: which tools are actually suited to radiomics?

This study provides a broad comparative evaluation of general-purpose and radiomics-specific AutoML frameworks, benchmarking 11 tools across 10 heterogeneous radiomics datasets spanning CT and MRI, multiple anatomical regions, cohort sizes and clinical classification tasks.

Rather than focusing solely on predictive accuracy, the comparison also considered computational efficiency, accessibility, software maturity and interpretability—key factors for practical adoption. Several radiomics-specific frameworks proved obsolete, difficult to use or computationally impractical, highlighting important gaps in the current AutoML landscape.

Among the evaluated solutions, Simplatab achieved the highest average AUC (78.46%) with a moderate 1.1-hour runtime, providing the strongest overall balance between predictive performance and computational efficiency. However, its predictive performance was not significantly superior to several more computationally intensive general-purpose alternatives.

The findings show both the promise and current limitations of AutoML in radiomics, identifying the need for more mature, user-friendly solutions that extend beyond model building toward fully automated, reproducible and clinically translatable radiomics workflows.