Add support for std

[tsoutsman]

The CI is too slow to even check out the rust submodule, let alone build the compiler. Probably best to create a separate branch rather than merging into theseus_main.

To upstream this we would have to:

• Implement the rest of std functionality
• Publish theseus_ffi and theseus_shim to crates.io

[tsoutsman]

Bump.

[kevinaboos]

yea, can't we use a typical conversion method for this? I know there are existing fat pointer types (pointers w/ metadata) in libcore, perhaps we can leverage that (or just DIY) to restrict the interface and types that can be used to convert to/from FatPointers

[kevinaboos]

Check out this section about strict provenance and the types included therein
https://doc.rust-lang.org/beta/core/ptr/index.html#strict-provenance

[kevinaboos]

e.g., https://doc.rust-lang.org/beta/core/primitive.pointer.html#method.to_raw_parts

[tsoutsman]

Well the problem is that the types in core aren't FFI-safe.

yea, can't we use a typical conversion method for this?

I mean the function (at least for moving from a fat pointer to a trait object) is ultimately unsafe. There's nothing guaranteeing that the implementation of the function on the other side of the FFI boundary is really returning a fat pointer corresponding to e.g. Writer.

[tsoutsman]

Is this safety doc correct?

[tsoutsman]

@kevinaboos should be ready for review

[kevinaboos]

@kevinaboos should be ready for review

Great! I'll take a look tonight or tomorrow late afternoon.

[kevinaboos]

So i know that in our mtg, you said that if we used something like stabby or abi_stable, we'd have to infect the rest of the codebase with their specific Arc-like types. But I wonder how it would be any different than defining our own types in theseus_ffi -- couldn't we just have the shim and libtheseus layers convert to/from those existing ABI-stable FFI types, regardless of whether they came from our lib or another lib? Seems like we could save the effort (and potential soundness/safety sharp edges) of implementing and maintaining our own FFI types by just using their types for that specific set of crates only.

My other major concern is mostly just the transmutes used everywhere, hopefully we can use existing fat pointer types with explicit to/from conversions instead.

[kevinaboos]

this can impl Send/Sync if and only if it actually comes from types that impl Send/Sync. Here I don't see that being the case yet, as it's always transmuted to/from something.

Instead, it'd be cleaner and safer to use a From/Into impl on limited types that you know are safe to convert to/from fat pointers like this. Given a set of closed conversion methods like that, then it might be okay to impl Send/Safe.

[kevinaboos]

yea, can't we use a typical conversion method for this? I know there are existing fat pointer types (pointers w/ metadata) in libcore, perhaps we can leverage that (or just DIY) to restrict the interface and types that can be used to convert to/from FatPointers

[kevinaboos]

Check out this section about strict provenance and the types included therein
https://doc.rust-lang.org/beta/core/ptr/index.html#strict-provenance

[kevinaboos]

what is this, something that the build of rust generates?

[tsoutsman]

Yes, we set the specific path to our custom rustc installation in .cargo/config.toml.

[tsoutsman]

couldn't we just have the shim and libtheseus layers convert to/from those existing ABI-stable FFI types, regardless of whether they came from our lib or another lib?

So not really. E.g. we call app_io::stdout and get an alloc::sync::Arc<dyn Writer>. We can't call stabby::Arc::from_raw(alloc::sync::Arc::into_raw(writer)) because stabby::Arc and sync::Arc could have different layouts. And there's not really much else we could do. We can't get an owned reference to the dyn Writer and so we can't turn it into a stabby trait object and wrap it in a stabby::Arc.

And conceptually that makes sense. writer is layed out in memory as an alloc::sync::Arc<dyn Writer> and there is no way we can interpret it as a stabby::Arc<vtable!(dyn Writer)> because they have different layouts.