Add tuned NLDD cases

[jakobtorben]

Add tuned cases that uses the NLDD non-linear solver. The most important aspect of the tuning has been to use the stronger non-linear solver to be able to take longer timesteps and control the timesteps better to reflect the stronger but more costly non-linear solver.

On some of my local tests this has given a significant speedup on the SPE11C case, compared to the default configurations in the repo and my own tuned Newton case. Some more benchmarking and tuning might be needed but this should be a good starting point to test the NLDD solver for SPE11.

[jakobtorben]

For the NLDD run, I got a speedup of 1.56 compared to one of my Newton runs with strict parameters. Completing a 168x120x100 simulation took 47 393 s on my 12 core desktop CPU.

The most important aspects for the SPE11C case have been:

• Reducing the target Newton iteration count to 2/3, to reflect the stronger but more costly non-linear iteration.
• Setting max Newton iterations to 6.
• Build OPM with NDEBUG such that it doesn’t crash if we get NaN but instead just throws a problem and cuts the timestep.
• Turned off oscillation detection and correction (this part is not 100 % verified to be smart yet).
• Increase the max timestep to 10 years to allow longer timesteps for the first 1000 years.
• Use the “newtoniteration” timestepper as the “pid-newtoniteration" has some bugs in it.
• For the linear solver I have used CPR_trueimpes instead of CPRW, since CPRW is not yet supported for NLDD. When we use source instead of wells they behave the same.
• I have tried partitioning based on uniform edge weights and trans edge weights, but I have not concluded on the best option yet.

I have been running with recent OPM master with the following pull requests added:

• Avoid deleting derivatives with zero pressure difference opm-models#877
• Set correct target iterations in SimpleIterationTimestepControl opm-simulators#5476
• Avoid deleting derivative i.e. getting "wrong" matrix in case of zero… opm-simulators#5236
• initialize water saturation for gas dissolved in water opm-models#908

This should make it fail less but I don’t they are critical for it to work.

For SPE11A I also tried using fewer target Newton iterations. I also increased the max number of NLDD iterations and linear solver iterations, since it was usually struggling with the linear solves. The result was that NLDD used around half of the timesteps but the timesteps used more linear iterations and therefore took longer, such that the overall simulation time was about the same. I have not had a chance to run the final tuned version submitted here.

For SPE11B I have not done any tests but I assume that similar options to SPE11C will work here.

Note that all the NLDD runs have been compared to my Newton simulations with a lot longer timesteps than what was default in the pyopmspe11 repo. The Newton case can also probably be tuned better so the comparison might not be 100 % fair. I have done all my tests on Cartesian grids but I believe it will be transferable to CP grids. I have not tried to run the SPE11A 1mm case as it takes too long, but hopefully the results from the 0.5 cm case should be transferable.