Comparison between Frequentist and Bayesian logistic regression on the example of real data - Fedorov, Mikhail (2024)
This study compares frequentist and Bayesian approaches to logistic regression using real-world data, with a particular focus on the impact of informative and non-informative priors in Bayesian analysis. The research is motivated by advances in computational capabilities that have made Bayesian methods increasingly accessible in machine learning applications. The methodology involved splitting the dataset into two parts: a primary subset (20%) for model evaluation and a larger auxiliary subset (80%) simulating prior knowledge. Three approaches were compared: classical logistic regression, Bayesian logistic regression with non-informative priors, and Bayesian logistic regression with informative priors. Model performance was evaluated using multiple metrics including Accuracy, Precision, Recall, F1-score, and ROC-AUC score, along with an analysis of 95% confidence intervals (CI) as well as credible intervals . Results demonstrate that while classical and non-informative prior Bayesian approaches showed similar performance, the Bayesian model with informative priors achieved superior results with an accuracy of 0.83 (compared to 0.78), F1-score of 0.85 (versus 0.81), and notably higher ROC-AUC of 0.93 (versus 0.81). The most significant improvement was observed in parameter uncertainty estimation, where the credible interval length undergone an average reduction of 15.1% compared to the classical approach, with improvements ranging from 10.3% to 28.2% across different variables.