CI: Windows flow takes 1.5h

[comphead]

Is your feature request related to a problem or challenge?

The windows CI flow consistently deteriorates and now takes 1.5h

Describe the solution you'd like

Investigate the reason

Describe alternatives you've considered

No response

Additional context

No response

[comphead]

With latest PR #13718

I can see some SLT tests are taking forever on Windows

Executed "map.slt". Took 460.5636141s
Executed "window.slt". Took 217.240136s
Executed "struct.slt". Took 1316.049712s
Executed "array.slt". Took 2407.7054095s

Which probably relates to different optimizations needed to apply for Windows platform comparing to unix/macos

[comphead]

The compilation is 13 mins for main DF and for CLI so 26mins in total comparing to 10mins in macos/unix, but the biggest problem for now is test run time which is probably related to different set of optimizations Windows expects

[findepi]

cc @sadboy

[comphead]

Thanks @korowa for pointing on slow Windows runner issues
actions/runner-images#7320

Some projects give up rust testing on Windows like DataDog/orchestrion#415

@alamb wondering should we pause the Windows flow temporarily? The build time is incredibly slow and we holding GH resources for way longer time

Leaving this issue opened, probably can be fixed with
actions/runner-images#10806

[alamb]

@alamb wondering should we pause the Windows flow temporarily? The build time is incredibly slow and we holding GH resources for way longer time

I think it is a good think to consider

The value of running Windows CI in my mind is that we ensure that people developing on windows can do so given a stable base. I wonder if there is some way we can pare back windows testing (like maybe compile with minimal features and do minimal testing, sqllogictests for example)?

[comphead]

The thing is the sqllogictests are enormously slow. Looks at that numbers

Executed "array.slt". Took 2407.7054095s

it depends slightly on LTO and optimizations but still most of sqllogictests are 70-100x times slower than linux/macos

[alamb]

The thing is the sqllogictests are enormously slow. Looks at that numbers

Executed "array.slt". Took 2407.7054095s

😮 that is crazy. I don't really know anything about building / optimizing for windows, but there must be some low hanging fruit there. 2400 seconds is like 40 minutes!

[comphead]

I have already played with different optimizations and there is still no progress. Namely:

• different LTO settings
• different opt-levels
• windows images
• linkers
• code-units
  and many more. The best for now is either leave it as is or pause the flow

[findepi]

i don't personally don't care how much CPU hours we spend on windows runners, but I do care about build latency (overall throughput). Thanks for working on this @comphead . I appreciate the effort. I understand we exhausted what's readily available to current project maintainers. I OK disabling tests of windows if we see these queue up (and this affecting build latency). If someone cares about windows dev-ex & stability, we'd appreciate contributions to make the CI usable.

[alamb]

I agree 100% about build latency, etc
I have started writing up several related tickets / organizing the work under the following epic

• [EPIC] A collection of items to improve developer / CI speed  #13813