free ram + ssd = raid for tmp2 = 20% speed increasing

[daydreamfarm]

My plotting machine has 64GB ram, 1TB m.2 SSD, i5 11400 CPU, Ubuntu

Typical plotting time is 4200s

I create a raid0 with 45G ram + 100G ssd for tmp2, and got my speed record at 3400s, roughly save 20% time.

iostat:

Device tps kB_read/s kB_wrtn/s kB_read kB_wrtn
nvme0n1 2009.68 249.5M 357.3M 6.4T 9.1T
md0 2663.51 319.5M 523.0M 8.2T 13.4T

Pros:
Almost 1/3 disc write were leveraged to ram.

Cons:
Maybe the 100GB ssd partition will burn ultra fast.

[desmond3699]

Running On Windows i think something wrong with your setting
i5-11400
32GB 3200MHZ CL22
2 x 1TB NVME 3.0
(Windows Raid 0 give me 3800 ~ 3900s)
Drive D : 1 x 1TB NVME 4.0 Slot
Drive E : 1 x 1TB NVME 3.0 Slot
Result 2800s~

[daydreamfarm]

My ram is 2400, and with a cheap m.2 :)
I read some post about the difference between 2400 and 3200, 30% or more.

[ihola777]

good point. Currently, it costs 2200 more seconds to finish the single plot on 2x1T SSD temp dir. planning to buy some cheap SSDs and it will save my ass if it still has a 20% improvement, more or less.

[thiagottjv]

I create a raid0 with 45G ram

How did you do that? How did you managed to make ram look like a block device to use on mdadm?

[daydreamfarm]

modprobe brd rd_nr=1 rd_size=2295760 max_part=1
echo "2" >/sys/module/raid0/parameters/default_layout
mdadm --create --verbose /dev/md0 --level=0 --raid-devices=2  /dev/ram0 /dev/nvme0n1p1
mkfs.xfs -m crc=0 /dev/md0
mount /dev/md0 /hybrid_raid

[thiagottjv]

Hi, thanks for the code. The only thing I changed was the rd_size to 50Gb.
I was using 2 2Tb 970 EVO in raid0.
I tried to put the ram as the 3rd raid part. I see the RAM beeing consumed, but it had no effect in the plot time for me.

[daydreamfarm]

If you try to combine a 45G ram with 2TB ssd, most rw will still on ssd. Try 50Gb ram +65gb ssd instead?

…

On Mon, Jun 21, 2021, 7:13 AM thiagottjv ***@***.***> wrote: Hi, thanks for the code. The only thing I changed was the rd_size to 50Gb. I was using 2 2Tb 970 EVO in raid0. I tried to put the ram as the 3rd raid part. I see the RAM beeing consumed, but it had no effect in the plot time for me. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#458 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AI3ZIXSVBTCUZ27GWMZTUKTTT5CG7ANCNFSM46VS3CKA> .

[thiagottjv]

You might be right.
I'll make 2 tests.
The first one will do a RAID0 with the 3 drives. two ssds with 150Gb each and the 50Gb on the RAM (and use this as tmp1 and tmp2, since using both on raid0 is faster then use it separate).
The second test will be a ssd for the temp1 and a RAID0 with 65gb on the SSD and 50Gb on the RAM for temp2.
In a couple of yours I'll post my findings here.

[thiagottjv]

@daydreamfarm Yeah... same thing... just a bit worse... one min or two...
I thought that I was finally using the rest of the RAM maybe comparing with only one SSD it would be better, but comparing with 2 ssds in raid0 it gets worse.
With both ssds in raid I was getting around 19 to 20min per plot on the first plot and 21.5 to 22min on the following plots.
With the RAM (from all the tests) I was getting 21 to 22.5min on the first plot.
I thing the best thing anyone can do is instead of buying large NVMEs is buying multiple small ones and use those PCI-e adapters.

Thanks for the help! I love to try all these variants!