RFC: extend Queue to support packed queue #39
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Liu Jiang <[email protected]>
The virtio queue is duplex comunication channel, it receives available descriptors from the driver and sends used descriptor to the driver. So the queue could be split into a receive endpoint and a sender endpoint. With split queue, the receiver endpoint receives available descriptors from driver by accessing the descriptor table and available ring. It also updates device event suppression status when receiving available descriptors. For the sender endpoint, it puts used descriptors into the used ring. It also queries driver event suppression status when sending used descriptors. For packed queue, it includes a descriptor table, a driver event suppression block and a device event suppression block. So packed queue could be abstracted in the same way as split queue. Current queue implementation is optimzed for single thread environment, there a single thread will receive available descriptors, handle these descriptors and then send those used descriptors in order. This mode will limit the performance of high IO virtio backends, such as virtiofs. So split the queue into receiver and sender. Due to implementation complex, the receiver still only supports single thread environment. For the sender endpoint, both a single thread version and a multiple thread version are provided. For the multi-thread version, it could be send between threads, and the sender and the receiver could work concurrently. Signed-off-by: Liu Jiang <[email protected]>
The Virtio 1.1 specification states that: The device MUST ignore the write-only flag (flags&VIRTQ_DESC_F_WRITE) in the descriptor that refers to an indirect table. So change Descriptor.is_writable() to follow the spec. Signed-off-by: Liu Jiang <[email protected]>
The main interface to access available descriptor chain is the DescriptorChain, so hdie internal helper methods of Descriptor. There are several constraints related descriptor flags, simply exposing accessors for those flags may cause misuse. So make them as private. It's also a preparation for packed queue. Signed-off-by: Liu Jiang <[email protected]>
The virtio 1.1 spec defines two types of descriptors, the traditional split descriptor and the new packed descriptor. So add definition for packed descriptor. Signed-off-by: Liu Jiang <[email protected]>
Enhance the DescriptorChain iterator to handle packed descriptor too. The SplitDescriptor is still used for external interface, and internal PackedDescriptor is converted to SplitDescriptor because the latter contains more information. Signed-off-by: Liu Jiang <[email protected]>
jiangliu
requested review from
lauralt,
bonzini,
sameo,
sboeuf and
andreeaflorescu
Change AvaiIter to an enum, so it could support both split and packed queues. Signed-off-by: Liu Jiang <[email protected]>
Pass more informtion to UsedRingT:set_ready() for packed queue. Signed-off-by: Liu Jiang <[email protected]>
Signed-off-by: Liu Jiang <[email protected]>
Refine the Queue::add_used() interface to support packed queue. Signed-off-by: Liu Jiang <[email protected]>
|
Hi @jiangliu ! I was wondering if you had the chance to run any performance tests on the packed queues implementation and if you noticed any significant improvement compared to split queues. |
I have no performance data yet. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes： #6
This patch set is based on #38 , and aims to implement an PoC by extending current Queue implementation to support Packed Queue. Current design shares most part with split queue, and use flag to control support of packed.
Currently it only implements basic support of packed to receive available descriptors and send used descriptors. More effort is needed to support device/driver event suppress mechanism.