Added DE optimizer

[adtserapio]

Hi Sir @dirkschumacher! In response to #57, I have added the DE optimizer! :)

[dirkschumacher]

Thanks. Would you mind adding a small test to make sure the optimzier works?

[adtserapio]

Hi Sir @dirkschumacher I tried adding a test and tried but then I keep getting this error. I have carefully examined the documentation and saw that the current example for SNE, SA, and SPSA should suffice, but I am running into some erors. :(

/Library/Frameworks/R.framework/Versions/3.6/Resources/library/
RcppEnsmallen/include/ensmallen_bits/function/static_checks.hpp:464:3:
error: static_assert failed due to requirement 
'CheckDecomposableEvaluate<OptimFunction, Mat<double>, Mat<double> >::value'
"The FunctionType does not have a correct definition of Evaluate(). 
Please check that the FunctionType fully satisfies 
the requirements of the NonDifferentiableDecomposableFunctionType API; 
see the optimizer tutorial for more details."

when I try to run the test

test_that("DE works", {

  optimizer = optimizer_DE(tolerance = -1)

  optimize <- compile_optimization_problem(
        data = list(),
        evaluate = function(x) {
          return(2 * norm(x)^2)
        },
        optimizer = optimizer_DE(tolerance = -1)
      )

      result <- optimize(matrix(c(1, -1, 1), ncol = 1))
      expect_true(sum(result) < 0.1)
})

Could you please advise me? Thank you very much Sir for your time!!

[dirkschumacher]

It seems the DE Optimizer wants a decomposable function. If that is the case, then the current implementation needs a bit more work and we would need to add that interface first, before one can implement the DE Optimizer.

From here

// For decomposable functions: return the number of parts the optimization
// problem can be decomposed into.
size_t NumFunctions();

// For decomposable objectives.  This should calculate the partial objective
// starting at the decomposable function indexed by 'start' and calculate
// 'batchSize' partial objectives and return the sum.
double Evaluate(const arma::mat& parameters,
                const size_t start,
                const size_t batchSize);

[adtserapio]

Forgive my naïvite, to implement this, I've checked the rcppsmallen package to include a NumFunctions() and

double Evaluate(const arma::mat& parameters,
                const size_t start,
                const size_t batchSize);

based on the PR https://github.com/coatless/rcppensmallen/pull/2/files. Do you think there is a why to borrow the functionality to implement this feature in armacmp. @dirkschumacher what do you think would be the steps to implement this.

[dirkschumacher]

At a high level I think there are some steps necessary:

• compile_optimization_problem needs to have an optional num_functions parameter that takes a function returning an int.
• Then all optimizers need to be of a proper class (S3 maybe or R6), not just a C++ string. That class can generate the necessary C++ code and also nows if the function is decomposable.
• compile_optimization_problem uses the optimizer classes to generate code. It also signales an error if necessary functions for the optimizers are not supplied.

[dirkschumacher]

I close this PR. If you want to continue working on it, let me know and I reopen it.