- randomly perform full weight update while doing PEFT
- heuristically perform full weight update while doing PEFT
- if the variance of the PEFT gradients are above some threshold, run the step again to perform a full weight update
Dec 08, 2024
Dec 08, 2024
Dec 08, 2024
Dec 08, 2024