You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: _posts/2025-03-12-hardware-plugin.md
+5-5
Original file line number
Diff line number
Diff line change
@@ -35,7 +35,7 @@ Before introducing the vLLM Hardware Plugin, let's first look at two prerequisit
35
35
36
36
Based on these RFCs, we proposed [[RFC] Hardware Pluggable](https://github.com/vllm-project/vllm/issues/11162), which integrates the `Platform` module into vLLM as a plugin. Additionally, we refactored `Executor`, `Worker`, `ModelRunner`, `AttentionBackend`, and `Communicator` to support hardware plugins more flexibly.
37
37
38
-
Currently, the vLLM team, collaborate with vLLM Ascend developers, has successfully implemented the Platform module introduced in the RFC. We also validated the functionality through the [vllm-project/vllm-ascend](https://github.com/vllm-project/vllm-ascend) project. Using this plugin mechanism, we successfully integrated vLLM with the Ascend NPU backend.
38
+
Currently, the vLLM team, collaborate with vLLM Ascend developers, has successfully implemented the Platform module introduced in the RFC. We also validated the functionality through the [vllm-project/vllm-ascend](https://github.com/vllm-project/vllm-ascend)and [vllm-project/vllm-spyre](https://github.com/vllm-project/vllm-spyre) projects. Using this plugin mechanism, we successfully integrated vLLM with the Ascend NPU and IBM Spyre backends.
39
39
40
40
---
41
41
@@ -55,7 +55,7 @@ You can refer to the [`platform.py`](https://github.com/vllm-project/vllm-ascend
55
55
56
56
#### Step 2: Implement Custom Worker, Model Runner, Attention Backend, and Communicator Modules
57
57
58
-
Depending on the new backend’s requirements, implement the following modules:
58
+
Depending on the new backend's requirements, implement the following modules:
59
59
60
60
```python
61
61
from vllm.worker.worker_base import WorkerBase
@@ -64,11 +64,11 @@ from vllm.attention.backends.abstract import AttentionBackend
64
64
from vllm.distributed.device_communicators.base_communicator import CommunicatorBase
65
65
```
66
66
67
-
Each of these classes has a corresponding base class in vLLM. Again, you can refer to [vLLM Ascend’s implementation](https://github.com/vllm-project/vllm-ascend/tree/main/vllm_ascend) for an example.
67
+
Each of these classes has a corresponding base class in vLLM. Again, you can refer to [vLLM Ascend's implementation](https://github.com/vllm-project/vllm-ascend/tree/main/vllm_ascend) for an example.
68
68
69
69
#### Step 3: Register the Plugin
70
70
71
-
Register the plugin in `setup.py` using Python’s entry point mechanism:
71
+
Register the plugin in `setup.py` using entrypoint mechanism of python:
72
72
73
73
```python
74
74
setup(
@@ -132,7 +132,7 @@ INFO 02-06 15:49:01 __init__.py:181] Platform plugin ascend is activated
132
132
133
133
---
134
134
135
-
## What’s Next?
135
+
## What's Next?
136
136
137
137
Moving forward, we will continue collaborating with developers in the vLLM community to enhance the following aspects:
0 commit comments