Changing the optimizer

Hello,

From other questions I understand that we cannot really modify the model and the loss function.
Can we modify the optimizer?
Also, can we use another initialization method than the default one?

Thanks