torch.optim.Adam | |
torch.optim.AdamW | Adam with decoupled weight decay |
torch.optim.Adamax | A variant of Adam, based on infinity norm |
torch.optim.SparseAdam | Lazy version of Adam |
torch.optim.NAdam | Adam with first moment estimate replaced by Nesterov Momentum |
torch.optim.RAdam | Rectified Adam |