Which propositions about shrinkage methods in linear regression are true?

The assertion that the selection of a tuning parameter can be addressed with cross-validation is indeed accurate. In the context of shrinkage methods in linear regression, such as lasso and ridge regression, cross-validation is often employed to determine the optimal value of the tuning parameter, typically denoted as λ. This parameter controls the strength of the penalty applied to the regression coefficients. By using cross-validation, practitioners can evaluate model performance across different subsets of the data, which helps in selecting a tuning parameter that minimizes prediction error and enhances model generalization to new data.

With regard to shrinkage methods, as λ increases, the penalty term in lasso and ridge regression imposes more constraint on the model coefficients, which can lead to a reduction in their values. However, higher values of λ can eventually saturate the effect and lead to coefficients tending towards zero. Therefore, the idea that the penalty term has no effect as λ increases is misleading, as it undersells the importance of the penalty in managing coefficient estimates.

The comparison between lasso and ridge also has nuances. Lasso regression tends to produce sparser solutions compared to ridge regression, meaning it can effectively reduce the number of variables included in the model by setting some coefficients to zero. However, this does not