Make it sense to introduce further optimiser?

[Anselmoo]

@haifeng-jin would it make sense to implement further KERAS Optimizer or just over-complicated the optimization process?

autokeras/autokeras/graph.py

Lines 292 to 322 in 424c0cc

	if optimizer_name == "adam":
	optimizer = keras.optimizers.Adam(learning_rate=learning_rate)
	elif optimizer_name == "sgd":
	optimizer = keras.optimizers.SGD(learning_rate=learning_rate)
	elif optimizer_name == "adam_weight_decay":
	steps_per_epoch = int(self.num_samples / self.batch_size)
	num_train_steps = steps_per_epoch * self.epochs
	warmup_steps = int(
	self.epochs * self.num_samples * 0.1 / self.batch_size
	)
	lr_schedule = keras.optimizers.schedules.PolynomialDecay(
	initial_learning_rate=learning_rate,
	decay_steps=num_train_steps,
	end_learning_rate=0.0,
	)
	if warmup_steps:
	lr_schedule = keras_layers.WarmUp(
	initial_learning_rate=learning_rate,
	decay_schedule_fn=lr_schedule,
	warmup_steps=warmup_steps,
	)
	optimizer = keras_layers.AdamWeightDecay(
	learning_rate=lr_schedule,
	weight_decay_rate=0.01,
	beta_1=0.9,
	beta_2=0.999,
	epsilon=1e-6,
	exclude_from_weight_decay=["LayerNorm", "layer_norm", "bias"],
	)

I assume this could be added via new elif-conditions?

thx for feedback

[haifeng-jin]

We may add some common ones like in #1736.
However, if the user have special needs, we should recommend them to use KerasTuner.