Machine learning and other robust approaches at the service of survival analysis : Alternatives to selected methods in statistical inference and prediction - Štěpánek, Lubomír (2023)
Survival analysis is a popular field of statistics and deals with many tasks both in statistical inference and prediction. While comparison of survival curves as one of the typical inferential tasks is performed using the log-rank test and other approaches, prediction of time to an event of interest for a given individual is commonly made using Cox proportional hazard model or others. However, all the methods in the survival toolbox are limited by relatively strict statistical assumptions, which violations may bias the results of the techniques applied to real data. In this work, we address the issue that the commonly used methods, both in statistical inference and prediction, are limited by their assumptions, and improve them using robust approaches, particularly machine-learning algorithms and delta method. In general, machine-learning approaches do not require to meet so strict assumptions; that is the reason we may get more robust alternatives to the traditional techniques. While the log-rank test or Cox proportional hazards model (or others) might be used within statistical inference in survival analysis for comparing two or more groups represented by their survival curves, we investigate rather tree-based methods for the same task and derive some new statistical properties of this approach. Intuitively, a random forest containing a large proportion of trees with sufficient complexity, adjusted by tree pruning, can classify individuals from various groups into two or more classes depicted by their survival curves, which tends to reject the null hypothesis about no statistical difference between the curves. Thus, a proportion of trees with sufficient complexity classifying into two or more groups, depicted by their survival curves, is very close to the p-value estimate as an analogy of the classical Wald's t-test output of the Cox's regression. We denote the p-value's analogy as phi-value.