Yogi: Optimizer

$$v_t = \beta_2 v_t-1 + (1 - \beta_2) g_t^2$$

The name is actually an acronym derived from the mechanics of the update: Y et O ther G radient I nformation. yogi optimizer

Not implementing learning rate decay.

optimizer = tf.keras.optimizers.Yogi( learning_rate=0.01, beta_1=0.9, beta_2=0.999, epsilon=1e-3, # Note: Yogi uses a larger default epsilon (1e-3 vs Adam's 1e-7) initial_accumulator_value=1e-6 ) $$v_t = \beta_2 v_t-1 + (1 - \beta_2)