Add Configurable Page Size Policy to MmapRegion #127
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a
PageSizePolicy
enum which can be used to configure the behavior ofMmapRegion
instances upon creation. The current options are:BasePages
, which uses the defaultmmap
behaviorTransparentHugepages
, which uses THP viamadvise
ExplicitHugepages
, which passesMAP_HUGETLB
tommap
The page size policy is at a per-mapping granularity, so a given VM can have different policies for different mappings, if desired. A helper function,
from_ranges_with_policy
, allows the VMM to configure mapping policies when constructing mappings from address ranges.The behavior is stubbed out on the
mmap_windows.rs
implementation, but the arguments are received there to allow the file to compile.The intention of this PR is to implement the functionality required for hugepage support in vm-memory (for #118), but without invasively modifying the external API, pending further discussion. This is mostly accomplished by adding extra constructors/functions which take a PageSizePolicy as an argument, but leaving existing functions alone.
These changes were experimentally tested for Firecracker by modifying Firecracker's local fork of vm-memory to include similar changes. Empirically, we were able to boot an Alpine microvm up to ~37% faster, and an Ubuntu microvm up to ~30% faster, reduce on-boot page faults by ~70×, all with negligible extra memory overhead. We would like to implement these changes here, so that other VMMs derived from vm-memory can hopefully reap the same benefit.
A few design questions we're not too sure about:
What is the correct way to expose this configuration to VMMs? The way we've chosen is to add a special
from_ranges
function inmmap.rs
that accepts a policy per tuple. However, as of recently, Firecracker is using a slightly modified local fork of vm-memory to implement dirty bit tracking, in addition to this repo. The local fork complicates how (and where) classes are used/defined (for example, it overridesmmap.rs
), making it difficult to utilize these changes downstream. We're not sure how other downstreams of this repo use the APIs, either. Advice to improve the API is much appreciated.At which level of abstraction should this policy belong? We decided to place the enum in
mmap.rs
since paging policy is closely related to mmapped regions, and not so closely related to GuestMemory in general. However, by attaching a policy to all MmapRegions, we're requiring a Windows implementation (which, for now, is just a no-op). Would it make sense to implement anExplicitHugepages
policy for Windows, if the OS provides that kind of control?