Small change to test case not detected

[bdforbes]

If I make a very small change to a test case, e.g. changing the 3 below to a 4, it seems the change is not detected (when running via the testing pane), and I can keep running the test with results as if it was still 3. I need to make a more substantial change, like removing and re-adding the @test line, in order have the change detected.

@testitem "My tests" begin
    @test foobar() == 3
end

Is this a known issue, or by design, or something that can be configured?

[bdforbes]

Apologies, I posted this in completely the wrong repo.