Docker on Debian and Ubuntu : Error response from deamon

[andromeda979]

Dear all,

I would like to share with you some interesting facts observed testing the maximum load supportable by the application Praktomat while it is running in a Virtual server.

Afterwards, I want to point out the following lack of Debian 8. Namely, even installing the last releases, Docker (1.11.1) does not support more then 80-100 students connected to Praktomat, giving back the following error: “Error response from deamon: Cannot destroy container ……”. I want to underline that this error, it is not due to Praktomat itself, but only to a lacks of Docker. Even if I configured correctly Debian, using

/ etc / default / grub "GRUB_CMDLINE_LINUX_DEFAULT =" cgroup_enable = memory swapaccount = 1 "

as suggested, I still get back the following message:

WARNING: No kernel memory limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support

To this end I opened a post on Github. Here you can find the link: moby/moby#22245 (comment)

Also on Ubuntu 16.04 LTS some problems arise: The first one is caused by Docker (Tasks) and the second one by Threads PID. Nevertheless, this time I was able to fix them.
Running the command systemctl status docker, we can notice that Docker has a limit number of tasks of default. This implicates (possible) additional problems on maximum load in the server “Tasks: 47 (limit: 512)”. The problem can be resolved, by the following command:

systemctl set-property docker.service TasksMax=4096.

This sets the limit for the number of tasks to 4096 “Tasks: 47 (limit: 4096)”.

In order to fix the second problem, we have to modify the limit number of the Threads PID modifying /etc/systemd/system.conf and setting DefaultTasksMax=1000.

Clearly this load depends on the machine that it is available. For completeness, we want to remark that it is possible to set DefaultTasksMax=unlimited whenever the machine can support such load.

Hoping to help someone in trouble.

[mhecker]

Hello!

Many thanks for your observations and tips. We, as well, have encountered issues involving certain docker and linux kernel versions.
We are currently running Debian 8.4, but not with the default kernel-package, but a more-recent linux-4.5 kernel, and docker-engine 1.11.1-0~jessie.

Can you elaborate on your observation that "no more than 80-100 students connected to Praktomat" are supported? By "connected to Praktomat", do you mean: "simultaneously logged-in, some of them submitting solutions", or "80-100 solutions being submitted simultaneously"? Or are you running some (artificial) stress-test?

[andromeda979]

Hello!

Thank you for your message.

I observed that whenever 80-100 students are simultaneously connected to Praktomat in a Lab session (usually it lasts two hours), Praktomat returns the error that I previously pointed out. It is not clear with how many simultaneously uploading, the Docker crashes: A roughly estimating I would say less that 50 students. I did not explore all the details, because I am now changing the server.

I wrote this post because, even if the last release of Debian and of Praktomat are installed, the same error appears whenever 100 solutions are simultaneously uploaded. I want to point out that this result is obtained running Jmeter.
Switching to the last realise of Ubuntu and implementing the modification described in previous post, the Docker can support at least 200 simultaneously uploading (with a suitable configuration of Apache).

For any further questions, please do not hesitate to contact me again!

Hoping to be useful,

[mhecker]

Thanks for the report!

Are you you using the Ubuntu16.04 default kernel (apparently based on linux 4.4, see: http://packages.ubuntu.com/xenial/linux-image ), or a newer kernel?
IIRC, we had some problems with linux kernels older than 4.5 when running multiple checkers simultaneously (specifically, when selecting "run all checkers" on a task from the admin interface).