ACP: add `array::from_raw_parts()`

[daxpedda]

Proposal

Problem statement

There is no "better" way to unsafely cast a [T; N] to a [U; N] or a &[T; N] to a &[U; N] apart from just pointer casting or using mem::transmute().

The idea here is that APIs like Vec::from_raw_parts() and slice::from_raw_parts() are harder to misuse and clearer then pointer casts. If instead pointer casting is the recommended way, this proposal is moot.

The example includes a comparison between both.

Motivating examples or use cases

#[repr(transparent)]
struct NonZeroWrapper<T: YourTrait>(T);

impl<T: YourTrait> NonZeroWrapper<T> {
    pub fn batch_computation<const N: usize>(values: &[Self; N]) -> [Self; N] {
        // No way to avoid pointer casts or `transmute()`.

        let t_values: &[T; N] = unsafe { &*(values as *const [Self; N]).cast() };
        let t_result = T::batch_computation(t_values);
        unsafe { (&*ManuallyDrop::new(t_result) as *const [T; N] as *const [Self; N]).read() }
    }
    
    pub fn batch_computation_vec(values: &[Self]) -> Vec<Self> {
        // Safety requirements are clear and the only obvious mistake is forgetting to use `ManuallyDrop`.

        let t_values = unsafe { slice::from_raw_parts(values.as_ptr().cast(), values.len()) };
        // This could be improved with `Vec::into_raw_parts()`.
        let mut t_result = ManuallyDrop::new(T::batch_computation_vec(t_values));
        unsafe { Vec::from_raw_parts(t_result.as_mut_ptr().cast(), t_result.len(), t_result.capacity()) }
    }
}

Solution sketch

I believe adding a method directly transforming a [T; N] to a [U; N] or a &[T; N] to a &[U; N] should be covered by the "safe transmute API". Instead what I'm proposing is to add an equivalent of Vec::from_raw_parts() and slice::from_raw_parts() for fixed arrays.

Equivalent of Vec::from_raw_parts():

impl<T, const N: usize> [T; N] {
    pub unsafe fn from_raw_parts(ptr: *mut T) -> [T; N] {
        ...
    }
}

Equivalent of slice::from_raw_parts():

pub const unsafe fn from_raw_parts<'a, T, const N: usize>(
    data: *const T,
) -> &'a [T; N] {
    ...
}

pub const unsafe fn from_raw_parts_mut<'a, T, const N: usize>(
    data: *mut T,
) -> &'a mut [T; N] {
    ...
}

The naming is not ideal.

Alternatives

To convert &[T; N] to a &[U; N] it is currently possible to just use slice::from_raw_parts() and then to convert it to a &[U; N] via TryInto.

I'm not aware of any way to convert a [T; N] to a [U; N] without just pointer casting or mem;;transmute_copy(). The simplest way seems to be ptr::read(<*mut [T; N]>::cast()).

Alternatively it seems also possible to use <[_; N]>::map() or array::from_fn() to do all this safely. The compiler seems to optimize this away to a memcpy.

Links and related work

This problem was encountered in RustCrypto/traits#1896.

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[kennytm]

impl<T, const N: usize> [T; N] {
    pub unsafe fn from_raw_parts(ptr: *mut T) -> [T; N] {
        ...
    }
}

😕 ptr.cast::<[T; N]>().read() ? (which is basically the same as transmute_copy)

And if you mean -> *mut [T; N] it would be simply be ptr.cast().

[daxpedda]

😕 ptr.cast::<[T; N]>().read() ? (which is basically the same as transmute_copy)

And if you mean -> *mut [T; N] it would be simply be ptr.cast().

The "Motivating Example" and "Alternatives" section already includes currently possible ways to do this. Apologies if I'm misinterpreting.

I understand of course if the position of the team is that pointer casting is good enough.

---

I think my "Problem Statement" was very poorly written, which makes this look like I'm asking how to do this instead of how to improve it.

I've updated the OP to hopefully make more sense. But to clarify: the idea is to add an equivalent of slice::from_raw_parts() for fixed array slices and Vec::from_raw_parts() for owned arrays.

Dedicated functions like slice::from_raw_parts() have much tighter safety requirements and are therefor much harder to misuse. The whole premise of this proposal is that using these functions are preferred over pointer casting and mem::transmute(). If this is not the case and I'm mistaken, please let me know!

I hope these comparisons make sense: I'm perfectly aware that there is no UB-free way to construct a Vec from a raw pointer (and allocated with the corresponding allocator ...) without Vec::from_raw_parts(). The same really applies to slices: to construct a valid slice a length is required together with a pointer, so slice::from_raw_parts() is kind of a necessity, unlike this proposal for arrays which has alternatives.

[hanna-kruppe]

I don’t see how the functions proposed here have significantly simpler safety conditions than pointer cast + read or transmute_copy — they seem to leave all the hard parts to the caller anyway.

For a by-value transmute from [T; N] to [U; N] (which does have somewhat simpler preconditions) I’d just start by spelling it as array.map(|x| /* transmute x */) and hoping that the map optimizes as well as the copy implied by a whole-array transmute. In your motivating use case the “transmute x” part could even be safe.

For the by-reference version, the signature &[T; N] -> &[U; N] (forcing same N and same lifetime) is better because it at least reduces some boilerplate in the safety comment (only need to say why &T -> &U is fine). I’m not sure if that makes enough of a difference to be worth an API. In the long term, safe transmute APIs will probably replace most uses, and may also give better ways of expressing the not-entirely-safe cases (e.g., still handle the length and lifetime, but add a const parameter/bound for transmuting the elements).

Part of the challenge in your motivating use case (the by-value part) is that T is a type parameter so it can’t be transmuted to/from Wrapper<T> even individually. This isn’t the fault of the array, so a new API for arrays would only solve it accidentally by also including the element transmute.

[daxpedda]

For a by-value transmute from [T; N] to [U; N] (which does have somewhat simpler preconditions) I’d just start by spelling it as array.map(|x| /* transmute x */) and hoping that the map optimizes as well as the copy implied by a whole-array transmute. In your motivating use case the “transmute x” part could even be safe.

I can confirm that I have observed this being optimized away to a memcpy in every case I checked.

For the by-reference version, the signature &[T; N] -> &[U; N] (forcing same N and same lifetime) is better because it at least reduces some boilerplate in the safety comment (only need to say why &T -> &U is fine). I’m not sure if that makes enough of a difference to be worth an API. In the long term, safe transmute APIs will probably replace most uses, and may also give better ways of expressing the not-entirely-safe cases (e.g., still handle the length and lifetime, but add a const parameter/bound for transmuting the elements).

Indeed safe transmute APIs would address the showcased issue entirely.

Part of the challenge in your motivating use case (the by-value part) is that T is a type parameter so it can’t be transmuted to/from Wrapper<T> even individually. This isn’t the fault of the array, so a new API for arrays would only solve it accidentally by also including the element transmute.

i was imagining that with generic_const_exprs it would be somehow possible to enable mem::tranmute() for type parameters by being able to express the size constraints.

---

I don’t see how the functions proposed here have significantly simpler safety conditions than pointer cast + read or transmute_copy — they seem to leave all the hard parts to the caller anyway.

After reading your response I realized I was thinking about it wrongly. My thinking was that dereferencing any pointer, through ptr::read() or *, has very broad safety requirements. In comparison from_raw_parts() can be much more specific. But in hindsight this doesn't make much sense, because the safety requirements are the same, its just that the type narrows down a lot of what one needs to be careful about.

So I agree that it doesn't narrow safety requirements.

Putting it side by side, it only really improves expressiveness and makes it a bit easier to review, I think.

let t_values: ManuallyDrop<[T; N]> = ...;
let u_values: [U; N] = unsafe { t_values.as_ptr().cast::<[U; N]>().read() };

let t_values: &[T; N] = ...;
let u_values: &[U; N] = unsafe { &*t_values.as_ptr().cast() };

let t_values: ManuallyDrop<[T; N]> = ...;
let u_values: [U; N] = unsafe { <[U; N]>::from_raw_parts(t_values.as_ptr().cast()) };

let t_values: &[T; N] = ...;
let u_values: &[U; N] = unsafe { array::from_raw_parts(t_values.as_ptr().cast()) };

Not sure if that alone is value enough for Std and if the team agrees at all.

---

Thank you for looking into it. Feel free to close it if there's nothing else.

[the8472]

The problem statement talks about going from [T; N] to a [U; N], but the solution sketch goes from *mut T to [T; N] or &[T; N], i.e. no U -> T conversion.

Can you clarify which of those conversion operations you are proposing?

[daxpedda]

The problem statement talks about going from [T; N] to a [U; N], but the solution sketch goes from *mut T to [T; N] or &[T; N], i.e. no U -> T conversion.

Can you clarify which of those conversion operations you are proposing?

The problem statement and the solution sketch don't map 1:1. I'm specifically focusing on adding equivalents of Vec::from_raw_parts() and slice::from_raw_parts() for fixed arrays as a solution.

A more direct solution to the problem statement is the safe transmute API. From that perspective it could also be concluded that the problem statement already has a planned solution. Adding the proposed APIs would require a different problem that isn't solved by the safe transmute API.

To emphasize this again: the proposed API is supposed to be a better alternative to pointer casting.

I updated the OP again to hopefully better reflect my intentions.