Description
I tried this code:
fn main() {
#[derive(Debug)]
struct Old(i32);
impl Drop for Old {
fn drop(&mut self) {
if self.0 == 44 {
panic!();
}
println!("Dropped Old: {}", self.0);
}
}
#[derive(Debug)]
struct New(i32);
impl Drop for New {
fn drop(&mut self) {
println!("Dropped New: {}", self.0);
}
}
let vec: Vec<Old> = vec![Old(11), Old(22), Old(33), Old(44), Old(55)];
let vec_ptr = vec.as_ptr();
let mapped_vec: Vec<New> = vec.into_iter().map(|x| New(x.0)).take(2).collect();
let mapped_vec_ptr = mapped_vec.as_ptr();
println!("vec_ptr: {vec_ptr:?}");
println!("mapped_vec: {mapped_vec_ptr:?}");
println!("{mapped_vec:?}");
}I expected to see this happen:
The drop implementation for New(11) and New(22) is called.
Instead, this happened:
Everything that was mapped and the allocation was leaked.
Meta
rustc --version --verbose:
rustc 1.63.0 (4b91a6ea7 2022-08-08)
binary: rustc
commit-hash: 4b91a6ea7258a947e59c6522cd5898e7c0a6a88f
commit-date: 2022-08-08
host: aarch64-apple-darwin
release: 1.63.0
LLVM version: 14.0.5
I discovered this issue when I learned that IntoIterator re-uses a Vec allocation if possible. Which prompted me to read the code. Running the example with cargo miri run also reports the memory leak. And changing New(i32) to New(i64) no longer displays the memory leak. This is what I think happens:
[ 11| 22| 33| 44| 55] Values
[ 0 | 1 | 2 | 3 | 4 ] Indices
[Old|Old|Old|Old|Old] Start (default fn from_iter)
[New|Old|Old|Old|Old] Map [0] (fn collect_in_place)
[New|New|Old|Old|Old] Map [1] (fn collect_in_place)
[New|New|Drp|Old|Old] Drop [2] (fn forget_allocation_drop_remaining)
[New|New|Drp|Old|Old] Drop [3] (fn forget_allocation_drop_remaining) PANIC
While dropping [3] the ptr::drop_in_place(remaining) call in forget_allocation_drop_remaining panics.
Then [4] is being dropped, but honestly I don't really understand why. My understanding is that ptr::drop_in_place(remaining) should unwind. If anyone can explain this, please do so.
Then as part of unwinding collect_in_place IntoIter is dropped, but it has its fields set to an empty buffer of len 0, so it does nothing. Note, I suspect more optimal code gen could be achieved by using mem::forget to avoid generating code for the IntoIter drop that will never do anything after we cleared the fields.
The two created New will never get dropped, thus leading to the memory leak. Plus the main Vec allocation, previously owned by IntoIterator as stored in dst_buf will be leaked too. Because it was cleared in forget_allocation_drop_remaining and assumed to be taken over by let vec = unsafe { Vec::from_raw_parts(dst_buf, len, cap) }; but we never reached that point.
Proposed solution, a drop guard in collect_in_place that guards the forget_allocation_drop_remaining call and would drop the head in this case 0..2 and the backing allocation if forget_allocation_drop_remaining panics.