-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ZTS: allow alternate work and output dir #17059
Open
robn
wants to merge
7
commits into
openzfs:master
Choose a base branch
from
robn:zts-base-dir
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tonyhutter
reviewed
Feb 19, 2025
The operator can override TEST_BASE_DIR by setting its source var FILEDIR through zfs-tests.sh -d. There were a handful of cases where this was not honoured. By default FILEDIR (and so TEST_BASE_DIR) is /var/tmp, so there should be no functional change if the operator does nothing. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
The default file vdevs, constrained binpath and temporary runfiles were all explicitly places in /var/tmp. Instead, put them under FILEDIR, which is set from -d and defaults to /var/tmp. TEST_BASE_DIR is also initialised from FILEDIR, which means all data for the run will now end up under the operator-specified data dir. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
The config file value overrides any set by the operator, making it quite difficult to put the test output elsewhere. The default is /var/tmp/test_results (via BASEDIR in test-runner) so this shouldn't change anything for the default case. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
The default outputdir had a timestamp appended in TestRun.__init__, and then the timestamp was unconditionally applied again after the runfile had been loaded, assuming that an outputdir would be set in the runfile too. If the runfile didn't have an outputdir, then the outputdir would get a second timestamp appended. Further, if test groups or individual tests themselves specificed an outputdir, those would be set on their config, but would not get a timestamp appended. It's not entirely clear if that's wrong or not, but it is certainly not consistent with the rest. To clean all this up, change things to append a timestamp to a received outputdir (from arg or runfile) before setting it in any TestRun, TestGroup or Test object. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
Many tests use mktemp to create temporary files and dirs, which will usually put them in /tmp unless instructed otherwise. This had led to many tests trying to give mktemp a useful temp path in ad-hoc ways, and others just using it directly without knowing they're potentially leaving stuff lying around. So we set TMPDIR to FILEDIR, which makes the simplest uses of mktemp put things in the wanted work dir. Included here is a hack to get TMPDIR into the test. If a test has to be run as a different user (most of them), it is run through sudo. ld.so from glibc will not pass TMPDIR to a setuid program, so instead we re-set TMPDIR after sudo before running the target command. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
In all cases, rely on mktemp itself to make the best decision about where to place the file or directory. In all cases, that decision will be $TMPDIR, which we have set globally. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
Most of these are trying to use TMPDIR to put their work files somewhere sensible. Now that we've set up correctly, they can all just use mktemp to do the job. In a couple of places cleaning up temp files wasn't being done correctly, which has been fixed. Sponsored-by: https://despairlabs.com/sponsor/ Signed-off-by: Rob Norris <[email protected]>
tonyhutter
approved these changes
Feb 24, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
I realised that part of the reason my test rig takes about 15 hours for a full ZTS run is at least in part to do with most of the tests on file vdevs, on ext4, in a VM, on a zvol. Learned a few things, some patches coming soon etc. But the best thing is to just dig out a spare NVMe drive, direct attach it to the VM, and make ZTS use that for everything.
Turns out it's awkward to mount a separate drive at
/var/tmp
because a typical Linux wants to use that for some of its housekeeping and there's a mild boot ordering issue, so fine, I'll put it somewhere else. But then it turns out that some of the path handling throughzfs-tests
andtest-runner
and the tests themselves were a bit messed up, so it wasn't possible to get everything where I wanted it.So this PR makes it so that if you tell the test runners to use a different path, you get (mostly) get what you pay for. And now all the test work goes to
/zts
on my test rig, and a full run takes 4 hours, which is very much more manageable.Description
There's three separate changes here.
First, we make sure that the work dir (default
/var/tmp
) is threaded through properly. As before, it starts life as$FILEDIR
inzfs-tests.sh
, settable with-d
. That becomes$TEST_BASE_DIR
once it hits the test scripts. Some of those had/var/tmp
hardcoded, and have been updated. There's also a small handful of setup things like the default vdevs, theconstrained_path
command dir and the generated runfile for single tests that were also hardcoded, and now go to$FILEDIR
.Next, we remove the explicit
outputdir
from the runfiles, and properly thread throughtest-runner
's default of/var/tmp/test_results
. This was buggy; adding the timestamp suffix was not done consistently for all the different sources ofoutputdir
, so I've fixed that up. Not that we use it, but if the code is going to exist it should probably work right.Finally, I've made the use of
/tmp
and$TMPDIR
a lot more consistent. In many places, individual tests create work files (including large vdev backing files) using their best guess for a good temporary dir, but it was done very inconsistently and most often ended up with files in/tmp
, which I wanted to avoid. So I've made sure$TMPDIR
is set to$FILEDIR
/$TEST_BASE_DIR
, and updated uses ofmktemp
to use it (usually simplifying the call) and uses of$TMPDIR
to usemktemp
.Taken together, this brings the vast majority of the work output under the path specified by the operator in
zfs-tests.sh -d
, and I'm much happier.Worth noting that there's still a handful of outputs to
/tmp
, which I've left for now, mostly because I ran out of day and I'd rather get this posted than leave it lying around for weeks while I find time to finish it off properly.How Has This Been Tested?
Two successful full ZTS runs completed pointed to an alternate work dir. The handful of tests that were modified for
mktemp
/$TMPDIR
were run directly both on the default/var/tmp
and an alternate dir, all passed.Types of changes
Checklist:
Signed-off-by
.