bisync: the tests are too noisy

[ncw]

The bisync tests are too noisy.

rclone/rclone/cmd/bisync$ go test 2>&1 | wc -l
5527

When they go wrong in CI it is almost impossible to figure out what went wrong, eg https://github.com/rclone/rclone/actions/runs/8572078457/job/23493782434 - this is partly due to the hard to use web page which involves lots of scrolling within windows, but I think we can do better.

Ideally I'd like the tests to print nothing by default, except actual test failures. If we want to see more info then there is the -verbose flag (fstest.Verbose) which by default enables DEBUG logging, but in bisync's case you can use it to print as much stuff as you want.

What do you think @nielash ?

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

[nielash]

What do you think @nielash ?

I might be misunderstanding what you mean, but personally I find the verbosity pretty helpful, especially for the integration tests. I don't have access to most of those backends locally, so the -verbose flag wouldn't really help me in that case, as I'm just relying on the pub.rclone.org logs. If they don't have the debug info, test failures will be very hard for me to fix.

When they go wrong in CI it is almost impossible to figure out what went wrong, eg https://github.com/rclone/rclone/actions/runs/8572078457/job/23493782434

Not sure if you saw my comment from yesterday, but I found the issue pretty quickly -- the trick is to ctrl+F for the word miscompare. Then once you see what's different, the next step is usually to determine why it's different, and for that you usually need to see the debug log...

All of that said -- happy to make them quieter if they are bothering you! (But maybe we can delay this until I've fixed more of the test failures? As the logs are quite helpful for that.)

[ncw]

I might be misunderstanding what you mean, but personally I find the verbosity pretty helpful, especially for the integration tests. I don't have access to most of those backends locally, so the -verbose flag wouldn't really help me in that case, as I'm just relying on the pub.rclone.org logs. If they don't have the debug info, test failures will be very hard for me to fix.

The integration tests are run with -verbose so you don't have to worry there. -verbose logs are useful for debugging but terrible for quick overview!

When they go wrong in CI it is almost impossible to figure out what went wrong, eg https://github.com/rclone/rclone/actions/runs/8572078457/job/23493782434

Not sure if you saw my comment from yesterday, but I found the issue pretty quickly -- the trick is to ctrl+F for the word miscompare. Then once you see what's different, the next step is usually to determine why it's different, and for that you usually need to see the debug log...

I don't think we should need to search to find out why a test failed. I know new contributors find the test output very confusing so I'd like to make it simpler if we can.

All of that said -- happy to make them quieter if they are bothering you! (But maybe we can delay this until I've fixed more of the test failures? As the logs are quite helpful for that.)

👍