🚀[FEA]: Add the option to use external optimizers

[jarnaud2]

Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

Critical (currently preventing usage)

Please provide a clear description of problem you would like to solve.

I would like to request the addition of some functionality in Modulus to utilize external optimizers. Existing libraries can do this already, where an interface module is created to call external optimizer routines in other Python libraries, such as Scipy.

A specific example done in the Tensorflow library can be found at https://github.com/tensorflow/tensorflow/blob/v1.15.2/tensorflow/contrib/opt/python/training/external_optimizer.py. Here, the Tensorflow script calls the Scipy optimizer to minimize a particular function. The task would thus be to replicate this functionality with the PyTorch library, since it is what Modulus uses.

The motivation is for utilizing the L-BFGS-B optimizer in Scipy, which for some specific research problems, performs better than the PyTorch implementation.

Describe any alternatives you have considered

No response

[nuitlejour]

good suggestion!