Hyperparameter Optimization Using Grid Search and Random Search to Improve the Performance of Prediction Models with Decision Trees

Abstract

Hyperparameter selection to obtain optimal accuracy results is an important factor in improving model performance in data science. This study discusses a comparison of two hyperparameter optimization methods, namely Grid Search and Random Search, in the Decision Tree Classifier algorithm using the Breast Cancer Wisconsin (Diagnostic) Dataset from the UCI Machine Learning Repository. The dataset contains 569 samples with 30 numerical features describing the characteristics of breast cancer cells, such as mean radius, texture, perimeter, area, and smoothness, which are classified into two classes, namely malignant and benign. This study uses the CRISP-DM approach, which includes the stages of business understanding, data understanding, data preparation, modeling, and evaluation. In the modeling stage, three testing scenarios were conducted, namely the Decision Tree model without tuning, the model with Grid Search optimization, and the model with Random Search optimization. Performance evaluation was carried out using accuracy, precision, recall, and F1-score metrics. The results showed that hyperparameter optimization had a significant effect on model performance. The Decision Tree model without tuning produced an accuracy of 92.98%, while the model with Grid Search achieved the highest accuracy of 95.61%, and Random Search obtained an accuracy of 97.37%. Thus, it can be concluded that Grid Search provides the most optimal results in finding the best parameter combination, even though it requires longer computation time compared to Random Search.