Making tests pass if they fail

[NadChel]

Related: Rethrowing Throwables from EDT

I want to test my InvocationInterceptor that runs tests in EDT.

Among other things, it shouldn't swallow any Throwables. I wrote a test for that.

    @ExtendWith(EdtExtension.class) // my custom extension that I test
    class AnnotatedPerMethod {

        @Test
        @EdtRun // I test processing of this custom annotation
        void ifThrowablesThrown_theyGetRethrown_notSwallowed() {
            assumeTrue(EventQueue.isDispatchThread());
            throw new AssertionError();
        }

Cool, it fails.

But from the standpoint of my build, it should actually pass.

In other words, I need to write a test that passes if it fails.

How do I do it in JUnit 5 — which, unlike JUnit 4, does not support Test#expected?

Here's a simplistic MRE. You're not allowed to remove the throw. The error should actually get thrown from the test method. It means you can't wrap it in assertThrows(). The test sort of tests itself instead of testing something that happens inside the curly braces.

    @Test
    void shouldPass_byFailing() {
        throw new AssertionError();
    }

Java 8.

[mpkorstanje]

You'll want to have a look at the EngineTestKit. It does exactly what you need.

[NadChel]

I don't get it. If I need a separate test to test my successfully failing test, how do I avoid the execution and failure of the tested test (the one that should fail)?

[NadChel]

Also, how do I know which artifact version supports Java 8? One thing's for sure, 6.0.0-M2 is not one

UPD: Ok, 1.9.0 does it

[mpkorstanje]

I don't get it. If I need a separate test to test my successfully failing test, how do I avoid the execution and failure of the tested test (the one that should fail)?

You don't avoid the failure. You use (A) JUnit Jupiter to use the Engine Test Kit to run the (B) JUnit Jupiter test with your EdtExtension. Then you assert in A that B failed in the expected manner.

Also, how do I know which artifact version supports Java 8? One thing's for sure, 6.0.0-M2 is not one

You'd use the BOM with the latest v5 version.

https://docs.junit.org/current/user-guide/#running-tests-build

[sbrannen]

In other words, I need to write a test that passes if it fails.

That's a rather unusual requirement, but if you really need that you can implement a TestExecutionExceptionHandler.

See Exception Handling in the User Guide for details.

We also have an example @ExpectToFail annotation in our test suite that you can use for inspiration.

Though, to be honest, I would typically recommend the use of assertThrows() within the @Test method for such use cases.