AI Infra Repositories

A curated list of open source repositories for AI Engineers

If you want to contribute to this list, please make a Pull request

Contents

• AI Gateway
• AI Workload Manager
• Copilot Development
• Dataset Engineering
• Evaluation
• Fine Tuning
• Function Calling
• Graph RAG
• Guardrails
• Local Model Inference
• LLM Agent Framework
• Model Serving
• Observability
• Pre Training
• Prompt Engineering
• RAG Framework
• Structured Extraction
• Structured Generation
• Vector DB
• Voice Agent

AI Gateway

Name (site)	Github
Kong
Litellm
Portkey
RouteLLM
GPTRouter
tensorzero
missing studio

AI Workload Manager

Name (site)	Github
Ray
higgsfield
Dstack

Copilot Development

Name (site)	Github
copilotkit
OpenCopilot

Dataset Engineering

Name (site)	Github
label studio
CleanLab
Snorkel
Lilac
litdata

Evaluation

Name (site)	Github
Eval
RAGAS
Giskard
promptfoo
arize ai
uptrain
trulens
tonic-validate
LangEvals
Poyro

Fine Tuning

Name (site)	Github
Unsloth
ludwig
NeMo/Nvidia
LMFlow
litgpt
Axolotl
xTuring

Function Calling

Name (site)	Github
Gorilla
Tiger
NPI AI
empower
Composio
gorilla

Graph RAG

Name (site)	Github
graphRAG
cognee
FalkorDB
llm-graph-builder
tidb

Guardrails

Name (site)	Github
NeMo-Guardrails
Guardrails

Local Model Inference

Name (site)	Github
llama cpp
whisper cpp
ollama
gpt4all
private-gpt
open-interpreter
LocalAI
jan
dalai

LLM Agent Framework

Name (site)	Github
AutoGPT
MetaGPT
AgentGPT
Autogen
BabyAGI
CrewAI
SuperAGI
TaskingAI
LaVague
superagent
Rivet AI
AGiXT
cheshirecat
langroid
Griptape
Mirascope
Julep
ControlFlow
Dify
Flowise
langflow
Haystack
MemGPT
phidata
PromptFlow
chainlit
ragapp
burr

Model Serving

Name (site)	Github
vLLM
TensorRT-LLM
bentoml
lightllm
openfoundry
LitServe
truss

Observability

Name (site)	Github
pezzo
helicone
Portkey
Langfuse
langtrace
traceloop
trulens
log10
LangWatch
burr
hamilton

Pre Training

Name (site)	Github
Colossal AI
FastChat
pytorch-lightning
llm c
miniGPT

Prompt Engineering

Name (site)	Github
DSPy
Langfuse
hamilton

RAG Framework

Name (site)	Github
Langchain
LlamaIndex
FastGPT
ragflow
vanna
FlagEmbedding
llmware
llm-app
Cognita
casibase
SciPhi
trieve
autollm
canopy
NeumAI
WrenAI
ragna
Dify
Flowise
langflow
Haystack
MemGPT
phidata
PromptFlow
chainlit
ragapp
hamilton
RagBuilder

Security

Name (site)	Github
Armur

Structured Extraction

Name (site)	Github
Unstructured
omniparse
unstract
indexify
firecrawl
Scrapegraph-ai
extractous

Structured Generation

Name (site)	Github
guidance
outlines
instructor
jsonformer
sglang

Vector DB

Name (site)	Github
milvus
Qdrant
Chroma
Weaviate
pgvector
deeplake
txtai
marqo
lancedb
nucliaDB
oasysdb
lantern
vespa

Voice Agent

Name (site)	Github
vocode
bolna