Read Issue in writethrough mode

[llcooljayr]

My current setup is a flashcache that is in writethrough mode. I am puzzled by what occurs when I perform a read on a file that is cached. I would expect the read to come from the SSD not from both as indicated by the image. By the way, the files that i'm reading are fairly large if that helps. Thanks!

SSD (raid0) - sdc
HDD (raid5) - sdd

[mohans]

There may be portions of the file not cached on the SSD that will need to be read from disk. What is your cache hit rate as reported by flashcache (for the duration of the read of the file - you can zero the stats before the read).

On Wednesday, October 1, 2014 4:02 PM, llcooljayr [email protected] wrote:

My current setup is a flashcache that is in writethrough mode. I am puzzled by what occurs when I perform a read on a file that is cached. I would expect the read to come from the SSD not from both as indicated by the image. By the way, the files that i'm reading are fairly large if that helps. Thanks!
SSD (raid0) - sdc
HDD (raid5) - sdd
—
Reply to this email directly or view it on GitHub.

[llcooljayr]

Hi Mohans,

Thanks for your reply. I had some time to do further investigations

I started off with a clean/new cache setup for my testing.

I found out that the reason why I'm getting reads from the disk is because it was not caching my writes to begin with. My read_hit_ratio was 77%.

Isn't this a fundamental flaw of the write-through mode? I was under the understanding that writes are done to both SSD and HDD synchronously (for redundency hence safer). So when I perform a read, I should be reading from the SSD since the data is cached. I would expect to read the entire file from the SSD not from both. The result would make sense if the file I'm writing does not fit in the SSD or in a situation where the SSD is getting full and there is a need to do a replacement. But I don't understand why Im reading from both the SSD and HDD if there is enough space for my file in the SSD.

I think the root of the problem is when the write is performed. I was getting uncached_writes which is surprising since there is enough space in the SSD. As a result it causes the read to both SSD and HDD.

Thanks for your help.

[mohans]

Yes. If you start from a clean cache, your writes should not be uncached. Can you paste the output from 'dmsetup status' and 'dmsetup table' after your initial copy ?
I am curious to see why you see high uncached writes. Are your writes getting split up into < 4KB ios by the device mapper for some reason ?

 On Monday, October 6, 2014 7:04 PM, llcooljayr <[email protected]> wrote:

Hi Mohans,Thanks for your reply. I had some time to do further investigationsI started off with a clean/new cache setup for my testing.I found out that the reason why I'm getting reads from the disk is because it was not caching my writes to begin with. My read_hit_ratio was 77%.Isn't this a fundamental flaw of the write-through mode? I was under the understanding that writes are done to both SSD and HDD synchronously (for redundency hence safer). So when I perform a read, I should be reading from the SSD since the data is cached. I would expect to read the entire file from the SSD not from both. The result would make sense if the file I'm writing does not fit in the SSD or in a situation where the SSD is getting full and there is a need to do a replacement. But I don't understand why Im reading from both the SSD and HDD if there is enough space for my file in the SSD. I think the root of the problem is when the write is performed. I was getting uncached_writes which is surprising since there is enough space in the SSD. As a result it causes the read to both SSD and HDD.Thanks for your help.—
Reply to this email directly or view it on GitHub.

[llcooljayr]

Hi Mohans,

Thanks for your help. The data block size matters. Initially I was using 16k for my block size and got the result below (16k) after my initial copy. I destroyed my cache and restarted it with 4k block size. If my bs is 4k, I get 99% cache hits on my reads (which is what I want). However, using 4k as the block size allocates a lot of memory for the flashcache metadata which is not ideal in my situation. The increased of memory usage by the flashcache metadata was the reason why I increased the blocksize to 16k to reduce the amount of memory flashcache used for the metadata. Why do you think increasing the block size to larger than 4k have is resulting in unexpected results? Also I tried 8k block size and I was getting cached missed as well.

bs=16k

[mohans]

Hi

Yes. The flashcache block size should be 4KB (this is the default). Flashcache only caches 4KB blocks. If you configure a 16KB blocksize, the DM breaks up the IOs into smaller than 16KB (because of misalignment), flashcache therefore sees IOs that are < 16KB and does not cache these IOs.

Smaller than 4KB IOs are not recommended.

mohan

On Wednesday, October 8, 2014 7:04 PM, llcooljayr [email protected] wrote:

Hi Mohans,
Thanks for your help. The data block size matters. Initially I was using 16k for my block size and got the result below (16k) after my initial copy. I destroyed my cache and restarted it with 4k block size. If my bs is 4k, I get 99% cache hits on my reads (which is what I want). However, using 4k as the block size allocates a lot of memory for the flashcache metadata which is not ideal in my situation. The increased of memory usage by the flashcache metadata was the reason why I increased the blocksize to 16k to reduce the amount of memory flashcache used for the metadata. Why do you think increasing the block size to larger than 4k have is resulting in unexpected results? Also I tried 8k block size and I was getting cached missed as well.
bs=16k

—
Reply to this email directly or view it on GitHub.

[ghost]

Thank you for reporting this issue and appreciate your patience. We've notified the core team for an update on this issue. We're looking for a response within the next 30 days or the issue may be closed.