In distributed scenarios, how to configure k8s ports and which ports need to be opened?

[markluofd]

2 H800 node deploying DeepSeek R1 model

Using k8s deployment. If the pod only opens the server port and the port agreed by dist-init-addr, the worker will report an error and cannot connect to the master. I want to know which ports I need to open to the pod. Can it be configured?

[shuaills]

cc @ByronHsu

[shuaills]

Are you following https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3#example-serving-with-two-h2008-nodes-and-docker?

[markluofd]

Yes, I followed the instructions in the document. If I use docker --net host, there is no problem with 2-machine deployment. But now I want to use k8s. The network cannot be shared between k8s 2 pods. I need to specify the ports. If I only open 20000 and 40000, the service cannot start. It seems that the service also depends on many other ports, and these ports are random.

When I use docker --net host, I observe that the master node has many sglang processes that occupy many ports, and these port numbers are random.

[markluofd]

Sorry, I misunderstood. I thought the communication ports between k8s pods need to be configured in advance.

In fact, the ports between k8s pods are fully open by default, so there's no problem.

[Hugh-yw]

Sorry, I misunderstood. I thought the communication ports between k8s pods need to be configured in advance.

In fact, the ports between k8s pods are fully open by default, so there's no problem.

Did you deploy it through k8s deploy resources? How are the --dist-init-addr --node-rank 1 parameters defined?

[markluofd]

--dist-init-addr set to be master pod ip and port