Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optionally record exit status to a special expected output file #28

Open
sampsyo opened this issue Jul 8, 2023 · 0 comments
Open

Optionally record exit status to a special expected output file #28

sampsyo opened this issue Jul 8, 2023 · 0 comments

Comments

@sampsyo
Copy link
Contributor

sampsyo commented Jul 8, 2023

Currently, to specify an expected nonzero exit status for a test command, you have to configure it manually: either in turnt.toml or in an embedded marker in the test. This works fine when, for example, handcrafting test programs with expected failures.

But for differential testing (a newish use case for Turnt that is still not completely seamless), this is annoying. We instead want to save the expected exit status from one command run, in a file, and then compare it against the exit status from another command—just like we do with textual output streams.

Maybe we can hack this in with another special output value in the config, just like - and 2 indicate the stdout and stderr streams. Like this:

output.status = '?'

That would tell Turnt to save the exit status in a file called <testname>.status.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant