[Issue] "database locked" errors

[JoBilert]

Hello, I have installed CWA last week and followed your guide to the letter.

CWA started up fine and my existing library was showing up nicely.

I always get "database locked" errors.

[iroQuai]

So it's a different problem than in #55 (comment) ?

The database locked error suggests permissions aren't set correct. Is the database read/write accessible for the PGID/PUID you specified in the compose file?

[JoBilert]

To be honest - i am not sure if it is the same. I would say it is at least similar.

I am not on a SynologyNAS but the database is a mounted Samba/CIFS.sahre

Within the container the Folder is mounted like this:

drwxr-xr-x   2 abc  abc      0 Sep  1 22:43 calibre-library

and the database-file like this:

-rwxr-xr-x 1 abc abc 6156288 Aug 10 18:57 metadata.db

Outside the container I mounted it with PUID 1000 and GUID 1000:

drwxr-xr-x    2 t***r   t***r       0 Sep  1 20:43 books

-rwxr-xr-x 1 t***r t***r 6156288 Aug 10 16:57 metadata.db

(t***r is the correct name of the default user and group)

I hope that helps.

[crocodilestick]

Hi @JoBilert , when do these "database locked" errors occur? As in what triggers them while you are using CWA?

[JoBilert]

Whenever I try to write into the database. Reading from it works fine.

But I cannot delete books, edit metadata or add new books via "ingest".

[crocodilestick]

Can you run fuser /calibre-library/metadata.db to check if the DB is being used by another process which may lock it? Otherwise DB locks are managed by the OS, so copying your db to another directory, deleting the original, and replacing it with the copied one should remove any locks.

[JoBilert]

using fuser outside the container gives me the following result:

/home/t***r/books/metadata.db:
                     t***r    2017383 F.... python3

Inside the container I cannot use fuser as it is not available.

[JoBilert]

Copied the database out of the folder, deleted database and copied it back in.

No change.

Copied the database outside the folder and remapped the docker-volume Changes worked, but I obviously couldn't open any books anymore.

[PilgrimToHyperion]

I've been struggling with the exact same issue for 2 days. I set the I finally got mine working just now.

/books and /calibre-library folders and files are set like this after I copied my books into the /books folder and my metadata.db into the /calibre-library folder:

chmod 777 -R /books
chmod 777 -R /calibre-library
chown abc:users -R /books
chown abc:users -R /calibre-library

After changing those permissions I had to reconnect the database. This is the step I've not done until today.

Seems like it needs to be reconnected before it understands right changes. Hope it works for you and if if it does, instructions should be expanded on.

[sjduggan5]

Getting the same issue - whenever I copy a book into /cwa-book-ingest, I get the following error:

Traceback (most recent call last):
  File "runpy.py", line 198, in _run_module_as_main
  File "runpy.py", line 88, in _run_code
  File "site.py", line 47, in <module>
  File "site.py", line 43, in main
  File "calibre/db/cli/main.py", line 253, in main
  File "calibre/db/cli/main.py", line 40, in run_cmd
  File "calibre/db/cli/cmd_add.py", line 471, in main
  File "calibre/db/cli/cmd_add.py", line 236, in do_add
  File "calibre/db/cli/main.py", line 184, in run
  File "calibre/db/cli/main.py", line 171, in db
  File "calibre/db/legacy.py", line 191, in __init__
  File "calibre/db/legacy.py", line 46, in create_backend
  File "calibre/db/backend.py", line 492, in __init__
  File "calibre/db/schema_upgrades.py", line 18, in __init__
  File "calibre/db/backend.py", line 1147, in execute
  File "src/cursor.c", line 240, in resetcursor
apsw.BusyError: BusyError: database is locked

It then logs that it successfully removes from /cwa-book-ingest and adds to /calibre-library.

[new-book-detector] Added /app/calibre-web-automated/cwa-import/book.file to Calibre database
[new-book-detector]: Removing /app/calibre-web-automated/cwa-import/book.file from import folder...
[new-book-detector]: /app/calibre-web-automated/cwa-import/book.file successfully moved/converted, the Ingest Folder has been emptied and is ready

It does delete from /cwa-book-ingest but nothing is added to my library. I don't think it's a permissions issue, the directory I'm mounting as the volume for /calibre-library is owned by the same UID and GID that the container has.

[crocodilestick]

@JoBilert @sjduggan5 do either of you happen to be running anything else that might also be using the metadata.db? E.g. Fullfat Calibre or calibre-server?

[sjduggan5]

@JoBilert @sjduggan5 do either of you happen to be running anything else that might also be using the metadata.db? E.g. Fullfat Calibre or calibre-server?

Not that I'm aware of. Is there a good way to programmatically check?

[crocodilestick]

@sjduggan5 try using lsof on the db file from outside the container

[sjduggan5]

@sjduggan5 try using lsof on the db file from outside the container

I get a bunch of warnings about running stat() that look like this:

lsof: WARNING: can't stat() nsfs file system /run/docker/netns/c34d259a5cf1
      Output information may be incomplete.

After which it just spits out that python is using it:

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
python3 7094  me    9u   REG   0,39   413696 3728 /mnt/nas/media/books/metadata.db

[JoBilert]

@sjduggan5 try using lsof on the db file from outside the container

I get a bunch of warnings about running stat() that look like this:

lsof: WARNING: can't stat() nsfs file system /run/docker/netns/c34d259a5cf1
      Output information may be incomplete.

After which it just spits out that python is using it:

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
python3 7094  me    9u   REG   0,39   413696 3728 /mnt/nas/media/books/metadata.db

Same output for me

[crocodilestick]

Have either of you tried killing the process and seeing if the issue remains?

[Microices]

I wonder if this comment can help to further investigate the issue. I too am facing the same issue, and it only occurs when the mount is a network mount.

https://www.mobileread.com/forums/showpost.php?p=2231723&postcount=2

[sjduggan5]

Sorry for the delayed response - I tried killing the process and the issue still remains. Though the issue is definitely what @Microices said as the mount is a network mount (I'm sharing a ZFS pool via Samba).

[Microices]

Found a likely workaround, https://stackoverflow.com/questions/17303076/sqlite-cannot-open-network-file-programmatically-even-though-worked-before but still unsure if network drive is the root cause.
Also not sure how I can test it as such posting it here.

[Microices]

Another setting that might be helpful for this scenario.
Issues with SQLite and network shares

[crocodilestick]

@Microices so did changing the journal mode of the DB fix the issue for you?

[Microices]

@Microices so did changing the journal mode of the DB fix the issue for you?

No I have not tried it...

[TechnoMaverick]

I have a mounted share on a Synology, and have run into this problem several times with trying to host various services. SQLite really doesn't like shares, and I've tried many different options (NFS in a couple different settings, the journal mode being set to delete, using the IDs of the Synology share owner or the ones that are on the host, and probably some other things I'm forgetting.)

The only solution I've come to is trying to run this off of my usual host machine and then backing it up via rclone to my Synology. Books don't take up a ton of space, I just need to figure out how to get everything involved set up.

I'm pretty new to self hosting and don't have any other ideas, but thought I would throw this out there in case it helps someone. I wanted to make this work on a mounted share but even setting the config to being on the host machine and the ingest and library being on the Synology throws permission errors as I kinda expected.

EDIT: After playing around with the install, I managed to get it working off of the setup I normally have. I don't really know how, but if anyone wants my docker compose, and steps, I'd be happy to share.