Which method is best suited for estimating test error with minimal bias in regression models?

Leave-one-out cross-validation is particularly effective for estimating test error with minimal bias in regression models due to its comprehensive approach to using the available data. In this method, each observation in the dataset is used once as a test set while all other observations form the training set. This allows the model to be evaluated on all available data points.

As a result, leave-one-out cross-validation helps in achieving an unbiased estimator because it leverages nearly the entire dataset for training, ensuring that the evaluation is close to the true performance of the model in practice. By systematically removing one observation and retraining the model on the remaining data, it provides a strong estimate of how the model will perform on unseen data.

This contrast with methods like k-fold cross-validation, random split validation, and bootstrap estimation, which might not utilize each data point in the same exhaustive way, can lead to higher variability and potentially biased estimates of test error, especially on smaller datasets.