add script to move images from pool to file system #44
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Not sure if this is something you want to include, but I wrote a script that is useful when people switch from in-pool storage (default) of images to on-disk storage. It will decode the attributes, store the file on disk into the directory that's configured in settings.json and rewrite the pool. I used it to decrease the database size of ether/etherpad-lite#4642 as in recent Etherpad version the pool is written in every key revision, i.e. a pad with a long history of 50000 revisions that has 10MB of images in the pool has a size of
500*10MB = 5GB
.It has two limitations: It won't work for S3 and it will delete pools from key revisions in case an image attribute is rewritten. This is mainly due to the fact that the pool of a key revision does represent the pool at the time the revision was written. In case you only want to merge this when I fix this, I think I can do it, i.e. iterating over the pool of key revisions and rewrite the attributes when needed instead of deleting the whole pool).
Update: Forgot to mention that it will only consider specific mime types, so depending on what people whitelist in settings.json they also need to add those mime types to this script. In case you want me to remove all non-image mime types I can do that. (The reason I included them is that it's very easy for people to add several non-image mime types that'll be stored in the pool and the goal of the script is to reduce the database size.)
Update 2: Make backups! :-)