VaultGemma: The most capable differentially private LLM
Applying the scaling laws to build VaultGemma The Gemma models are designed with responsibility and safety at their core. This makes them a natural foundation for developing a production-quality, DP-trained model like VaultGemma. Algorithmic advancements: Training at scale The scaling laws we derived above represent an important first step towards training a useful Gemma model with DP. We used the scaling laws to determine both how much compute we needed to train a compute-optimal 1B paramete