Divergence

[lfyao]

Hello! I'm running estimates in parallel and I get these message (seen at the very bottom) when running and found that some estimates are close to 0 already. My understanding is that I should just specify a higher value for the samples in stan_opts and set a higher value for adapt_delta like below. I already tried 5000 but get the same problems does the value need to be much much bigger?

stan = stan_opts(samples = 5000
                       control = list(adapt_delta = 0.99))

Error messages:

There were 5 divergent transitions after warmup. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
to find out why this is a problem and how to eliminate them. - 

Bulk Effective Samples Size (ESS) is too low, indicating posterior means and medians may be unreliable.
Running the chains for more iterations may help. See
http://mc-stan.org/misc/warnings.html#bulk-ess - 

[seabbs]

Hi @leanfranzl,

Thanks for the Q.

Hard to know exactly without some idea of the data using on etc. but potentially you may see some benefit from increasing the number of warmup samples and the overall number of samples (here samples is the sum across all chains and not per chain as is more usual so 5000 is not that many). Increasing adapt_delta even further would also help but will substantially increase run times.

I would say that whilst divergent transitions are not ideal and obviously we would prefer not to have them a very low number as a percentage of overall samples may be acceptable. On the other hand they might indicate a mismatch between the model and your data but this is hard to tell without further details. Regardless a low effective sample size is not really okay and so here more samples are likely needed.

[seabbs]

I'd also be careful with the example defaults used for the report delay as these are strictly examples. Here is an example implementation I am using for forecasting in the ECDC which might help: https://github.com/epiforecasts/europe-covid-forecast/blob/master/rt-forecast/delays.R

[seabbs]

If my point about the data issues above is correct it may be that some difficulty in fitting is just inevitable at this time and given the current model formulation in some data streams with very sparse case loads.

[lfyao]

Yes we're using regional_epinow. Thank you for the insight on areas with very few cases I agree that this would cause some problems. Though we do sometimes observe divergence problems even for areas that consistently have a high number of cases.

Sorry for the oversight, we do based our report delays from the data. If we're doing it correctly

EpiNow2::estimate_delay(delays = data$report_delay,
                                             bootstraps = 100,
                                             samples = 1000)

[seabbs]

That looks sensible to me.

In our use we have also seen model fitting issues (i.e divergences, and sometimes convergence issues) usually this is linked to fairly noticeable issues in the case data (i.e where it is clearly not a smooth epidemic process) that EpiNow2 doesn't include sufficient adjustments for.

If you have some examples of time series that are causing you issues (perhaps the data + a plot) I am happy to take a look and see if there is any within model changes that would help. A possible solution in some edge cases is to increase the exactness of the gaussian process approximation (see ?gp_opts for details) as this allows for increasingly rapid changes in Rt but at the cost of increased compute.

[lfyao]

I see! I'll look into it after my attempt with more iterations, just might take a few days. Thank you for the help!