Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to row::get blob into suitable containers #1189

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

Krzmbrzl
Copy link
Contributor

@Krzmbrzl Krzmbrzl commented Jan 2, 2025

"Suitable" means that the soci::is_contiguous_resizable_container is specialized to have its value member yield true. This specialization is provided for std::string and std::vector<T> where sizeof(T) == sizeof(char).

Fixes #1173

"Suitable" means that the container uses contiguous storage in memory
and holds entries of type T where sizeof(T) == 1. If this is given, then
the blob's data can be copied into a new instance of such a container,
which is then returned in the row::get<>() call.

This restores the previous possibility of reading blobs into objects of
type std::string but also generalizes the concept to any suitable
container (such as std::vector<unsigned char>).

Note: The check for whether the given container object uses contiguous
object storage is based on its iterators being classified as
random-access iterators. This is not a perfect check as in principle
there could be random-access iterators that don't iterate a contiguous
memory region. However, for typical containers this is very unexpected
as this would typically lead to rather inefficient access patterns when
making use of the random-access capability.
A container that is wrongly determined to be contiguous will likely
result in a program crash due to a segfault.

Fixes SOCI#1173
Copy link
Member

@vadz vadz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing this but is checking for arbitrary random access iterators really needed? I'd say that string and vector are the only containers that you may ever want to read BLOBs into but if we want to be more generic, we could just specialize is_contiguous_byte_container for them by default but document that it be specialized for custom types too: it seems like it would be simpler and safer.

include/soci/type-holder.h Outdated Show resolved Hide resolved
include/soci/type-holder.h Outdated Show resolved Hide resolved
@Krzmbrzl
Copy link
Contributor Author

Krzmbrzl commented Jan 7, 2025

I'd say that string and vector are the only containers that you may ever want to read BLOBs into but if we want to be more generic, we could just specialize is_contiguous_byte_container for them by default but document that it be specialized for custom types too: it seems like it would be simpler and safer.

It would be safer, yes, but also much more hassle (compared to "it just works") to get working. What I have in mind is someone using e.g. Boost containers.

The current implementation relies on any container type being passed in to be defined sensibly, that is in analogy with the standard containers. I think as long as this is true, then the assumptions in this PR hold (including the implicitly assumed availability member functions size and resize). That's why I thought that it is probably safe to go the plug-and-play route I chose.

If you know of even a single commonly-ish used container that would pass my heuristic without having contiguous storage in memory, then we obviously can't use this. However, I was unable to come up with one 👀

@vadz
Copy link
Member

vadz commented Jan 7, 2025

I think any legacy custom container wouldn't have resize() (the ones I worked with definitely didn't) and I do think that we need to test for its presence using SFINAE if we keep this version. This could be done, of course, but IMO it's not worth it.

I'd really like to keep the code as simple as possible while still avoiding regressions (such as #1173) and just allowing reading a BLOB into a string would be good enough for me. Allowing reading it into a vector is even more enough, I'm not even sure we need a trait at all — but I'm ready to have it if you think it's worth it, just let's maximally simplify it.

OTOH I thought of something else: why don't we specialize soci_cast for blob itself? It looks like we should be able to copy blobs using it, and I think this may have been allowed before as well, wasn't it?

@Krzmbrzl
Copy link
Contributor Author

Krzmbrzl commented Jan 9, 2025

I'll add a trait and a default implementation for an accessor. This way, the code can be more or less fully customized if needed, but the logic on SOCI's side is minimal.
I'll specialize the trait for std::string and std::vector<T> where sizeof(T) == sizeof(char).

OTOH I thought of something else: why don't we specialize soci_cast for blob itself? It looks like we should be able to copy blobs using it, and I think this may have been allowed before as well, wasn't it?

BLOBs are inherently non-copyable and I believe they always have been. Making Blobs copyable might be possible but I think this would involve a significant amount of work as different backends have different semantics of what a Blob actually represents and how they have to be accessed. At the very least the Blob handle (where present) would have to have shared_ptr like semantic to ensure that we don't have double-frees. However, copying Blobs would then only copy the SOCI Blob instance. Yet both instances would still point to the same "physical" Blob in the DBs. That's because for most backends, SOCI Blobs have more of a pointer-like semantic. Yet, this is not true for all backends (e.g. MySQL).

So all in all I believe that the Blob backends are just too different to support this with a consistent API and with consistent semantics.

@Krzmbrzl Krzmbrzl force-pushed the fix-blob-get-as-string branch from fb1fa42 to f2e60f3 Compare January 9, 2025 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reading BLOB from SQLite3 no longer working
2 participants