How do you make torchtune deterministic?

[Sinestro38]

Aside from setting the seed, how do you enforce torchtune to use deterministic algos under the hood with torch, set the cudnn deterministic flag, and appropriately set the CUDA workspace config.

[Sinestro38]

cc: @krammnic If you could give a pointer that would be very much appreciated :)

[krammnic]

Hey! You may use tune cp to copy the required recipe and modify it as you wish (in this case freeze seed and other required parameters how it can be done usually). Unfortunately, some parts might be not always deterministic due the floating point operations, but still results should be consistent