Skip to content

Commit af48ad2

Browse files
Document Baseten integration
1 parent 9ec878f commit af48ad2

File tree

2 files changed

+133
-0
lines changed

2 files changed

+133
-0
lines changed
Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Baseten
2+
3+
[Baseten](https://baseten.co) is an AI inference provider that supports open-source, fine-tuned, and custom models of all modalities. The Strands Agents SDK can be used to run against powered by any OpenAI-compatible LLM hosted on Baseten.
4+
5+
Baseten offers two options for LLM inference, both of which are compatible with the Strands Agents SDK:
6+
7+
- **Model APIs**: Access to pre-deployed models like DeepSeek and Llama
8+
- **Dedicated Deployments**: Custom model deployments with dedicated infrastructure
9+
10+
## Installation
11+
12+
Baseten is configured as an optional dependency in Strands Agents. To install, run:
13+
14+
```bash
15+
pip install 'strands-agents[baseten]'
16+
```
17+
18+
## Configuration
19+
20+
### API Key
21+
22+
You'll need a Baseten API key to use the service. Set it as an environment variable:
23+
24+
```bash
25+
export BASETEN_API_KEY="your-api-key-here"
26+
```
27+
28+
### Model APIs
29+
30+
Model APIs provide access to popular models through a common API endpoint:
31+
32+
```python
33+
from strands.models.baseten import BasetenModel
34+
35+
# DeepSeek R1 model
36+
model = BasetenModel(
37+
model_id="deepseek-ai/DeepSeek-R1-0528",
38+
client_args={
39+
"api_key": "your-api-key",
40+
},
41+
)
42+
43+
# DeepSeek V3 model
44+
model = BasetenModel(
45+
model_id="deepseek-ai/DeepSeek-V3-0324",
46+
client_args={
47+
"api_key": "your-api-key",
48+
},
49+
)
50+
51+
# Llama 4 Maverick model
52+
model = BasetenModel(
53+
model_id="meta-llama/Llama-4-Maverick-17B-128E-Instruct",
54+
client_args={
55+
"api_key": "your-api-key",
56+
},
57+
)
58+
59+
# Llama 4 Scout model
60+
model = BasetenModel(
61+
model_id="meta-llama/Llama-4-Scout-17B-16E-Instruct",
62+
client_args={
63+
"api_key": "your-api-key",
64+
},
65+
)
66+
```
67+
68+
**Available Model APIs:**
69+
70+
* `deepseek-ai/DeepSeek-R1-0528`: DeepSeek R1 0528 model
71+
* `deepseek-ai/DeepSeek-V3-0324`: DeepSeek V3 0324 model
72+
* `meta-llama/Llama-4-Maverick-17B-128E-Instruct`: Llama 4 Maverick 17B model
73+
* `meta-llama/Llama-4-Scout-17B-16E-Instruct`: Llama 4 Scout 17B model
74+
75+
### Dedicated Deployments
76+
77+
Dedicated deployments provide custom model hosting with dedicated infrastructure:
78+
79+
```python
80+
from strands.models.baseten import BasetenModel
81+
82+
deployment_id = "dq4kr413" # Your deployment ID
83+
environment = "production" # Environment (default: "production")
84+
base_url = f"https://model-{deployment_id}.api.baseten.co/environments/{environment}/sync/v1"
85+
86+
model = BasetenModel(
87+
model_id=deployment_id,
88+
base_url=base_url,
89+
environment=environment,
90+
client_args={
91+
"api_key": "your-api-key",
92+
},
93+
)
94+
```
95+
96+
**Environment Options:**
97+
98+
* `production`: Production environment (default)
99+
* `staging`: Staging environment
100+
* `development`: Development environment
101+
102+
## Usage
103+
104+
After installing `strands-agents[baseten]`, you can import and initialize the Strands Agents' Baseten provider as follows:
105+
106+
```python
107+
from strands import Agent
108+
from strands.models.baseten import BasetenModel
109+
110+
# Initialize model
111+
model = BasetenModel(
112+
model_id="deepseek-ai/DeepSeek-V3-0324",
113+
client_args={"api_key": "your-api-key"},
114+
)
115+
116+
# Create agent
117+
agent = Agent(model=model)
118+
119+
# Chat with the model
120+
response = agent("Hello! How are you?")
121+
print(response)
122+
```
123+
124+
Baseten models
125+
126+
127+
## References
128+
129+
* [Baseten Documentation](https://docs.baseten.co/)
130+
* [Strands SDK Documentation](https://docs.strands.ai/)
131+
132+

mkdocs.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@ nav:
8585
- Model Providers:
8686
- Amazon Bedrock: user-guide/concepts/model-providers/amazon-bedrock.md
8787
- Anthropic: user-guide/concepts/model-providers/anthropic.md
88+
- Baseten: user-guide/concepts/model-providers/baseten.md
8889
- LiteLLM: user-guide/concepts/model-providers/litellm.md
8990
- LlamaAPI: user-guide/concepts/model-providers/llamaapi.md
9091
- MistralAI: user-guide/concepts/model-providers/mistral.md

0 commit comments

Comments
 (0)