Modeling lapse rate in life insurance : A study of data balancing methods - Bogdan, Adam (2025)
Lapse rates have generally been estimated on product to product base. Using this approach, however, may be imprecise as it doesn't reflect each customers probabilities and therefore may negatively impact the cash flow predictions. Using real data from a Czech insurance company, this paper is aimed at generating individual probabilities as opposed to the estimated lapses for the whole product. Models like logistic regression, XGB, lasso, ridge, elastic net and random forest are estimated, tested and compared. The inevitable problem of unbalanced data is solved by over-sampling and/or under-sampling methods and their effects on predictions is deeply studied, thus more than 70 balanced training datasets are created. The main finding are that based on the provided dataset the various classification methods perform at fairly similar level. Since the balancing methods are based on K-Nearest Neighbors, the varying values of K were tested and it can be concluded that their parametrization does not significantly improve models. Comparing the main over-sampling methods, ADASYN methods had a higher F1 score than SMOTE methods. Also using hybrid methods did not prove to significantly improve prediction results compared to the individual methods. Overall, even the best models, do not perform at a level that would be deemed useful in real world use, judging by F1 scores that were at best around 0,2.