The SuperSONIC project implements server infrastructure for inference-as-a-service applications in large high energy physics (HEP) and multi-messenger astrophysics (MMA) experiments. The server infrastructure is designed for deployment at Kubernetes clusters equipped with GPUs.
The main components of SuperSONIC are:
- Nvidia Triton inference servers
- Dynamic muti-purpose Envoy Proxy:
- Load balancing
- Rate limiting
- GPU saturation prevention
- Token-based authentication
- (optional) Load-based autoscaling via KEDA
- (optional) Prometheus instance (deploy custom or connect to existing)
- (optional) Pre-configured Grafana dashboard
- (optional) OpenTelemetry Collector and Grafana Tempo for advanced monitoring.
helm repo add fastml https://fastmachinelearning.org/SuperSONIC
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm install <release-name> fastml/supersonic --values <your-values.yaml> -n <namespace>
To construct the values.yaml file for your application, follow Configuration guide.
The full list of configuration parameters is available in the Configuration reference.
| CMS | ATLAS | IceCube | |
|---|---|---|---|
| Purdue Geddes | ✅ | - | - |
| Purdue Anvil | ✅ | - | - |
| NRP Nautilus | ✅ | ✅ | ✅ |