[bug]: Webadmin interface stops working after some days until server is restarted

[oddlama]

What happened?

When I leave my server running for some days (I don't know exactly how long, but several days) the webadmin interface breaks and only shows a white screen when accessed. The server logs indicate that the files have gone missing (see log output below).

I have not touched any configuration related to the webadmin interface, I basically just used the default and configured some certificates, set the database to sqlite and added my custom SQL directory. Certainly nothing related to the webadmin. I can provide the full config if necessary.

How can we reproduce the problem?

I can reproduce the problem by just starting the server and waiting. To me this suggests that there might be a periodic cleanup of some sorts happening in the background that deletes these files, although since I don't know where they are stored I cannot confirm this.

Version

v0.9.x

What database are you using?

SQLite

What blob storage are you using?

SQLite

Where is your directory located?

SQL

What operating system are you using?

Linux

Relevant log output

Sep 22 13:43:40 linux stalwart-mail[386305]: 2024-09-22T11:43:40Z ERROR Resource error (resource.error) listenerId = "http", localPort = 8080, remoteIp = 127.0.0.1, remotePort = 41288, reason = "No such file or directory (os error 2)", path = "webadmin-c27017567ab36cc9_bg.wasm", causedBy = crates/common/src/manager/webadmin.rs:52
Sep 22 13:43:40 linux stalwart-mail[386305]: 2024-09-22T11:43:40Z ERROR Resource error (resource.error) listenerId = "http", localPort = 8080, remoteIp = 127.0.0.1, remotePort = 41272, reason = "No such file or directory (os error 2)", path = "webadmin-c27017567ab36cc9.js", causedBy = crates/common/src/manager/webadmin.rs:52
Sep 22 13:43:40 linux stalwart-mail[386305]: 2024-09-22T11:43:40Z ERROR Resource error (resource.error) listenerId = "http", localPort = 8080, remoteIp = 127.0.0.1, remotePort = 41270, reason = "No such file or directory (os error 2)", path = "output-94a36bb96862d625.css", causedBy = crates/common/src/manager/webadmin.rs:52

Code of Conduct

• I agree to follow this project's Code of Conduct

[dantefromhell]

I've been seeing this too sporadically. There's some HTML/ JSON output being rendered in my browser but I don't have the full string at hand since it's been a while since the last crash.

[mdecimus]

Stalwart unpacks the webadmin to the system's temporary folder. Perhaps you have a process deleting the temporary folders?

[oddlama]

Stalwart unpacks the webadmin to the system's temporary folder

Oh I see, that's likely the problem. Not sure if you are familiar with systemd, but it includes a timer that periodically cleans up temporary directories, and by default, it removes anything that hasn’t been accessed recently or is still open by a process. I assume stalwart doesn't keep an open file descriptor for the temporary files? That would explain why this is happening. systemd typically cleans up /tmp after 10 days and /var/tmp after 30 days, so any systemd-based installation with automatic /tmp cleanup might run into this if the web interface isn’t accessed during that time.

I’m not sure the webadmin files fit the typical idea of "temporary files" (i.e., storage used for short-lived operations). Depending on whether you want the files to be removed when the service stops, a runtime directory in /run or a cache directory in /var/cache might be a better solution to prevent this issue.

Whether this is a bug in stalwart or something for each distribution’s package maintainer to address might come down to interpretation. It would be great if stalwart could either switch to a more suitable directory or at least allow configuration of this directory, so the problem can be avoided downstream.

You might also find this article on safe usage of tmpfiles interesting.

[mdecimus]

@oddlama Just added a webadmin.path setting to specify the unpack directory.