FEAT: [WIP] Xavier: Share KV cache between VLLM replicas#2732
Draft
ChengjieLi28 wants to merge 11 commits intoxorbitsai:mainfrom ChengjieLi28:feat/share_kv_cache_vllm
+1,494-19
Commits
Commits on Jan 3, 2025
- committed
- committed
Commits on Jan 6, 2025
Commits on Jan 7, 2025
- committed
- committed
- committed
- committed