You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**RunGPT** is an open-source _cloud-native_large-scale **_multimodal models_** (LMMs) serving framework.
17
+
**RunGPT** is an open-source _cloud-native_**_large-scale language models_** (LLMs) serving framework.
18
18
It is designed to simplify the deployment and management of large language models, on a distributed cluster of GPUs.
19
-
We aim to make it a one-stop solution for a centralized and accessible place to gather techniques for optimizing large-scale multimodal models and make them easy to use for everyone.
19
+
We aim to make it a one-stop solution for a centralized and accessible place to gather techniques for optimizing LLM and make them easy to use for everyone.
20
20
21
21
22
22
## Table of contents
23
23
24
24
-[Features](#features)
25
-
-[Supported models](#supported-models)
26
25
-[Get started](#get-started)
27
26
-[Build a model serving in one line](#build-a-model-serving-in-one-line)
RunGPT provides the following features to make it easy to deploy and serve **large multi-modal models** (LMMs) at scale:
32
+
RunGPT provides the following features to make it easy to deploy and serve **large language models** (LLMs) at scale:
34
33
35
-
- Support for multi-modal models on top of large language models
36
34
- Scalable architecture for handling high traffic loads
37
35
- Optimized for low-latency inference
38
36
- Automatic model partitioning and distribution across multiple GPUs
@@ -41,35 +39,9 @@ RunGPT provides the following features to make it easy to deploy and serve **lar
41
39
42
40
## Updates
43
41
44
-
-**2023-05-12**: 🎉We have released the first version `v0.0.1` of RunGPT. You can install it with `pip install run_gpt_torch`.
42
+
-**2023-08-22**: The OpenGPT is now renamed to RunGPT. We have also released the first version `v0.1.0` of RunGPT. You can install it with `pip install rungpt`.
43
+
-**2023-05-12**: 🎉We have released the first version `v0.0.1` of OpenGPT. You can install it with `pip install open_gpt_torch`.
45
44
46
-
## Supported Models
47
-
48
-
<details>
49
-
50
-
RunGPT supports the following models out of the box:
51
-
52
-
- LLM (Large Language Model)
53
-
54
-
-[LLaMA](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/): open and efficient foundation language models by Meta
55
-
-[Pythia](https://github.com/EleutherAI/pythia): a collection of models developed to facilitate interpretability research by EleutherAI
56
-
-[StableLM](https://github.com/Stability-AI/StableLM): series of large language models by Stability AI
57
-
-[Vicuna](https://vicuna.lmsys.org/): a chat assistant fine-tuned from LLaMA on user-shared conversations by LMSYS
58
-
-[MOSS](https://txsun1997.github.io/blogs/moss.html): conversational language model from Fudan University
59
-
60
-
- LMM (Large Multi-modal Model)
61
-
62
-
-[OpenFlamingo](https://github.com/mlfoundations/open_flamingo): an open source version of DeepMind's [Flamingo](https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model) model
63
-
-[MiniGPT-4](https://minigpt-4.github.io/): aligns a frozen visual encoder with a frozen LLM, Vicuna, using just one projection layer.
64
-
65
-
For more details about the supported models, please see the [Model Zoo](./MODEL_ZOO.md).
66
-
67
-
</details>
68
-
69
-
70
-
## Roadmap
71
-
72
-
You can view our roadmap with features that are planned, started, and completed on the [Roadmap discussion](https://github.com/jina-ai/rungpt/discussions/categories/roadmap) category.
73
45
74
46
## Get Started
75
47
@@ -78,7 +50,7 @@ You can view our roadmap with features that are planned, started, and completed
78
50
Install the package with `pip`:
79
51
80
52
```bash
81
-
pip install run_gpt_torch
53
+
pip install rungpt
82
54
```
83
55
84
56
### Quickstart
@@ -269,8 +241,8 @@ dictionaries which contain role and content. For example:
269
241
import requests
270
242
271
243
messages = [
272
-
{"role": "user", "content": "Hello!"},
273
-
]
244
+
{"role": "user", "content": "Hello!"},
245
+
]
274
246
275
247
response = requests.post(
276
248
"http://localhost:51000/chat",
@@ -626,4 +598,4 @@ We welcome contributions from the community! To contribute, please submit a pull
626
598
627
599
## License
628
600
629
-
Rungpt is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
601
+
RunGPT is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.
0 commit comments