Issue with using django_gcp and django_reverse_admin - save order causes MissingBlobError

[morphogencc]

Support request

About me

Hello! I'm using django to make the backend for science museum installations. I use django to manage data, allow museum staff to make updates to multimedia on the museum floor, and ultimately provide the backend for visitor-facing multimedia experiences.

Request

• I've searched the list of existing issues
• I expected the docs to cover it in [this section](please provide link to where you'd expect to find help on this in the docs) but they don't cover it.

I'm trying to both django_gcp and django_reverse_admin to update content in my database. Right now, my parent models with a BlobField update just fine, but the ForeignKey model's BlobFields can't save and always give me a MissingBlobError:

Internal Server Error: /admin/mesoamerica/site/64/change/
Traceback (most recent call last):
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django_gcp\storage\operations.py", line 52, in copy_blob
    destination_blob = source_bucket.copy_blob(
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\google\cloud\storage\bucket.py", line 1910, in copy_blob
    copy_result = client._post_resource(
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\google\cloud\storage\client.py", line 627, in _post_resource
    return self._connection.api_request(
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\google\cloud\storage\_http.py", line 72, in api_request
    return call()
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\google\cloud\_http\__init__.py", line 494, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.NotFound: 404 POST https://storage.googleapis.com/storage/v1/b/my-museum-multimedia/o/_tmp%2Fd1263eb1-0469-4ea4-ace0-272d8ecdefba/copyTo/b/my-museum-multimedia/o/mesoamerica%2Fimages%2FCh1-e8c868fa-f800-47d7-8b67-0e0d0676f2de.png?prettyPrint=false: No such object: 
my-museum-multimedia/_tmp/d1263eb1-0469-4ea4-ace0-272d8ecdefba      

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\core\handlers\exception.py", line 55, in inner
    response = get_response(request)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\core\handlers\base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)   
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\contrib\admin\options.py", line 688, in wrapper
    return self.admin_site.admin_view(view)(*args, **kwargs)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\utils\decorators.py", line 134, in _wrapper_view
    response = view_func(request, *args, **kwargs)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\views\decorators\cache.py", line 62, in _wrapper_view_func
    response = view_func(request, *args, **kwargs)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\contrib\admin\sites.py", line 242, in inner
    return view(request, *args, **kwargs)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django_reverse_admin\__init__.py", line 207, in change_view
    return self._changeform_view(request, object_id, form_url, extra_context) 
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django_reverse_admin\__init__.py", line 275, in _changeform_view
    self._save_object(request, new_object, form, formsets, add)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django_reverse_admin\__init__.py", line 214, in _save_object
    self.save_related(request, form, formsets, change=not add)
  File "D:\Projects\myproject-django-cms\projects\cms\mesoamerica\admin.py", line 56, in save_related
    with transaction.atomic():
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\db\transaction.py", line 307, in __exit__
    connection.set_autocommit(True)
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\db\backends\base\base.py", line 501, in set_autocommit
    self.run_and_clear_commit_hooks()
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django\db\backends\base\base.py", line 779, in run_and_clear_commit_hooks
    func()
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django_gcp\storage\fields.py", line 255, in on_commit_valid
    copy_blob(
  File "D:\Projects\myproject-django-cms\projects\cms\env\lib\site-packages\django_gcp\storage\operations.py", line 86, in copy_blob
    raise MissingBlobError(
django_gcp.exceptions.MissingBlobError: Could not complete copy: source blob _tmp/d1263eb1-0469-4ea4-ace0-272d8ecdefba does not exist in bucket my-museum-multimedia

My guess is that django-gcp and django-reverse-admin aren't playing nice and objects are somehow getting saved / validated multiple times, after the temporary file has initially been moved. As this is an inter-plugin issue, I know this might not be a django-gcp issue, but I thought I'd reach out to see if you had ideas on how to resolve this.

Supporting Octue

Octue is an Open-Source Software organisation helping you and many others to fight climate change. Providing support for your issues costs us c. 500GBP/day. Might your organisation be in a position to help sponsor this issue?

• We already are (eg via technical partnership or GitHub Sponsors )
• Yes (we'll prioritise this issue, and be in touch)
• No (we'll work on this issue as and when it aligns with our core development activity)

[thclark]

Hi @morphogencc could you make a sketch out (enough for us to reproduce) of your models, your admin definition and your project dependencies please? I can then let you know how long this might take and see if we can reach a sponsorship agreement!

Thanks,

Tom

[thclark]

Alternatively: PRs are welcomed!