idea training randomly perform full weight update while doing PEFT heuristically perform full weight update while doing PEFT if the variance of the PEFT gradients are above some threshold, run the step again to perform a full weight update