Why would my cloud deploy get stuck installing python dependencies when I have made no changes?

[dagsir[bot]]

Does anyone experience weird issues with cloud deployments being stuck on installing python dependencies from pip?
Did not change any dependencies but seeing the issue since yesterday evening or so.

#13 60.70   Downloading json5-0.9.4-py2.py3-none-any.whl (17 kB)
#13 62.51   Downloading json5-0.9.3-py2.py3-none-any.whl (17 kB)
#13 64.31   Downloading json5-0.9.2-py2.py3-none-any.whl (27 kB)
#13 66.11   Downloading json5-0.9.1-py2.py3-none-any.whl (27 kB)
#13 67.91   Downloading json5-0.9.0-py2.py3-none-any.whl (27 kB)
#13 69.70 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See <https://pip.pypa.io/warnings/backtracking> for guidance. If you want to abort this run, press Ctrl + C.
#13 69.70 INFO: pip is looking at multiple versions of httptools to determine which version is compatible with other requirements. This could take a while.
#13 69.71 Collecting httptools&gt;=0.5.0
#13 69.73   Downloading httptools-0.6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
#13 69.74      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.7/441.7 kB 63.9 MB/s eta 0:00:00
#13 93.11   Downloading httptools-0.5.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (427 kB)
#13 93.12      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 427.8/427.8 kB 73.1 MB/s eta 0:00:00
#13 116.4 INFO: pip is looking at multiple versions of h11 to determine which version is compatible with other requirements. This could take a while.
#13 116.5 Collecting h11&gt;=0.8
#13 116.5   Downloading h11-0.13.0-py3-none-any.whl (58 kB)
#13 116.5      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.2/58.2 kB 18.3 MB/s eta 0:00:00
#13 186.9   Downloading h11-0.12.0-py3-none-any.whl (54 kB)
#13 186.9      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.9/54.9 kB 18.0 MB/s eta 0:00:00
#13 233.5 INFO: pip is looking at multiple versions of httptools to determine which version is compatible with other requirements. This could take a while.
#13 257.0   Downloading h11-0.11.0-py2.py3-none-any.whl (54 kB)
#13 257.0      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.6/54.6 kB 17.2 MB/s eta 0:00:00
#13 327.9   Downloading h11-0.10.0-py2.py3-none-any.whl (53 kB)
#13 327.9      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.7/53.7 kB 15.9 MB/s eta 0:00:00
#13 351.4 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See <https://pip.pypa.io/warnings/backtracking> for guidance. If you want to abort this run, press Ctrl + C.
#13 398.8   Downloading h11-0.9.0-py2.py3-none-any.whl (53 kB)
#13 398.8      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.6/53.6 kB 19.0 MB/s eta 0:00:00
#13 469.7   Downloading h11-0.8.1-py2.py3-none-any.whl (55 kB)
#13 469.7      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.8/55.8 kB 17.6 MB/s eta 0:00:00
#13 540.8   Downloading h11-0.8.0-py2.py3-none-any.whl (55 kB)
#13 540.9      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 55.6/55.6 kB 17.4 MB/s eta 0:00:00
#13 611.4 INFO: pip is looking at multiple versions of h11 to determine which version is compatible with other requirements. This could take a while.
#13 611.4 INFO: pip is looking at multiple versions of graphene to determine which version is compatible with other requirements. This could take a while.
#13 611.4 Collecting graphene&gt;=3
#13 611.4   Downloading graphene-3.2.2-py2.py3-none-any.whl (125 kB)
#13 611.4      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.6/125.6 kB 36.3 MB/s eta 0:00:00
#13 965.5 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See <https://pip.pypa.io/warnings/backtracking> for guidance. If you want to abort this run, press Ctrl + C.
#13 1178.8   Downloading graphene-3.2.1-py2.py3-none-any.whl (125 kB)
#13 1178.9      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.1/125.1 kB 5.8 MB/s eta 0:00:00
#13 1728.2   Downloading graphene-3.2-py2.py3-none-any.whl (124 kB)
#13 1728.3      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 124.6/124.6 kB 6.4 MB/s eta 0:00:00
#13 2279.0   Downloading graphene-3.1.1-py2.py3-none-any.whl (121 kB)
#13 2279.0      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 kB 6.0 MB/s eta 0:00:00
#13 2831.6   Downloading graphene-3.1-py2.py3-none-any.whl (114 kB)
#13 2831.7      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 114.2/114.2 kB 6.0 MB/s eta 0:00:00
#13 3389.9   Downloading graphene-3.0-py2.py3-none-any.whl (112 kB)
#13 3389.9      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 112.9/112.9 kB 5.9 MB/s eta 0:00:00
#13 3390.0 INFO: pip is looking at multiple versions of gql[requests] to determine which version is compatible with other requirements. This could take a while.
#13 3390.0 Collecting gql[requests]&gt;=3.0.0
#13 3390.0   Downloading gql-3.4.0-py2.py3-none-any.whl (65 kB)
#13 3390.0      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65.2/65.2 kB 17.5 MB/s eta 0:00:00
#13 5062.8 INFO: pip is looking at multiple versions of graphene to determine which version is compatible with other requirements. This could take a while.
#13 7319.6   Downloading gql-3.3.0-py2.py3-none-any.whl (63 kB)
#13 7319.6      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 63.4/63.4 kB 3.7 MB/s eta 0:00:00
#13 8449.9 INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See <https://pip.pypa.io/warnings/backtracking> for guidance. If you want to abort this run, press Ctrl + C.```

---

*The question was originally asked in [Dagster Slack](https://dagster.slack.com/archives/C02LJ7G0LAZ/p1700155335795719?thread_ts=1700155335.795719&cid=C02LJ7G0LAZ).*

[alangenfeld]

Depending on how you have specified the version requirements for your dependencies, new versions of the packages you depend on (directly or transitively) may have been published which can effect the resolution of packages. Specifying more exact versions of your dependencies can help avoid this problem.

[mudravrik]

Thanks for the reply! Yeah, transitive dependencies were my initial idea as well.
So the first thing I tried since posting the issue was simple local run of pip install -r requirements.txt --no-cache-dir in fresh clean venv. Just to check if it is something wrong with dependencies or not.
Appears, It works fine without any significant delays. So either it is not a correct way to reproduce what is going on inside pip installation in the runner or something else happened.

So right now I working on local running the workflow in act but without any success yet.

[mudravrik]

Well, I managed to reproduce the behaviour locally in act and, more importantly, in plain venv as well. Have to downgrade pip version to 22.0.2 which I believe is used to build dependencies inside the workflow.
So yeah, you are totally right, it should be some issues inside our dependencies and I should play with versions to fix the initial problem.
Thank you for help!