Pinned Loading

1. xlite-dev/lite.ai.toolkit xlite-dev/lite.ai.toolkit Public

   🛠 A lite C++ toolkit: Deploy 100+ AI models (Stable-Diffusion, Face-Fusion, YOLO series, Det, Seg, etc) via MNN, ORT and TRT. 🎉🎉

   C++ 4k 737

2. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 43.5k 6.6k

3. xlite-dev/Awesome-LLM-Inference xlite-dev/Awesome-LLM-Inference Public

   📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism etc. 🎉🎉

   Python 3.8k 265

4. PaddlePaddle/FastDeploy PaddlePaddle/FastDeploy Public

   ⚡️An Easy-to-use and Fast Deep Learning Model Deployment Toolkit for ☁️Cloud 📱Mobile and 📹Edge. Including Image, Video, Text and Audio 20+ main stream scenarios and 150+ SOTA models with end-to-end…

   C++ 3.1k 477

5. xlite-dev/CUDA-Learn-Notes xlite-dev/CUDA-Learn-Notes Public

   📚Modern CUDA Learn Notes: 200+ Tensor/CUDA Cores Kernels, FA2, HGEMM via MMA and CuTe (~99% TFLOPS of cuBLAS/FA2 🎉).

   Cuda 3.2k 342

6. xlite-dev/ffpa-attn-mma xlite-dev/ffpa-attn-mma Public

   📚FFPA(Split-D): Yet another Faster Flash Prefill Attention with O(1) SRAM complexity large headdim (D > 256), ~2x↑🎉vs SDPA EA.

   Cuda 160 7