You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: spring-ai-docs/src/main/antora/modules/ROOT/pages/api/aiclient.adoc
+43-38Lines changed: 43 additions & 38 deletions
Original file line number
Diff line number
Diff line change
@@ -1,20 +1,18 @@
1
+
[[AiClient]]
1
2
= AiClient
2
3
3
-
== Overview
4
-
5
-
The AiClient interface streamlines interactions with xref:concepts.adoc#_models[AI Models].
6
-
It simplifies connecting to various AI Models — each with potentially unique APIs — by offering a uniform interface for interaction.
4
+
The `AiClient` interface streamlines interactions with xref:concepts.adoc#_models[AI Models].
5
+
It simplifies connecting to various AI Models -— each with potentially unique APIs -— by offering a uniform interface for interaction.
7
6
8
7
Currently, the interface supports only text-based input and output.
9
-
You should expect some of the classes and interfaces to change as we support for other input and output types is implemented.
8
+
You should expect some of the classes and interfaces to change as we add other input and output types.
10
9
11
-
The design of the AiClient interface centers around two primary goals:
10
+
The design of the `AiClient` interface centers around two primary goals:
12
11
13
-
1. *Portability*: It allows easy integration with different AI Models, allowing developers to switch between differing AI models with minimal code changes.
12
+
* *Portability*: It allows easy integration with different AI Models, letting developers switch between differing AI models with minimal code changes.
14
13
This design aligns with Spring's philosophy of modularity and interchangeability.
15
14
16
-
17
-
2. *Simplicity*: Using companion classes like `Prompt` for input encapsulation and `AiResponse` for output handling, the `AiClient` interface simplifies communication with AI Models. It manages the complexity of request preparation and response parsing, offering a direct and simplified API interaction.
15
+
* *Simplicity*: By using companion classes like `Prompt` for input encapsulation and `AiResponse` for output handling, the `AiClient` interface simplifies communication with AI Models. It manages the complexity of request preparation and response parsing, offering a direct and simplified API interaction.
18
16
19
17
== API Overview
20
18
@@ -36,13 +34,11 @@ public interface AiClient {
36
34
37
35
The `generate` method with a `String` parameter simplifies initial use, avoiding the complexities of the more sophisticated `Prompt` and `AiResponse` classes.
38
36
39
-
40
37
=== Prompt
41
-
In a real-world application, it will be most common to use the generate method, taking a `Prompt` instance and returning an `AiResponse`.
38
+
In a real-world application, it is most common to use the `generate` method, taking a `Prompt` instance and returning an `AiResponse`.
42
39
43
40
The `Prompt` class encapsulates a list of `Message` objects.
44
-
Below is a truncated version of the Prompt class, excluding constructors and other utility methods:
45
-
41
+
The following listing shows a truncated version of the Prompt class, excluding constructors and other utility methods:
46
42
47
43
```java
48
44
public class Prompt {
@@ -57,7 +53,6 @@ public class Prompt {
57
53
58
54
The `Message` interface encapsulates a textual message, a collection of attributes as a `Map`, and a categorization known as `MessageType`. The interface is defined as follows:
59
55
60
-
61
56
```java
62
57
public interface Message {
63
58
@@ -70,15 +65,14 @@ public interface Message {
70
65
}
71
66
```
72
67
73
-
The Message interface has various implementations corresponding to the categories of messages that an AI model can process.
68
+
The `Message` interface has various implementations that correspond to the categories of messages that an AI model can process.
74
69
Some models, like OpenAI's chat completion endpoint, distinguish between message categories based on conversational roles, effectively mapped by the `MessageType`.
75
70
76
-
77
-
For instance, OpenAI recognizes message categories for distinct conversational roles such as "system", "user", or "assistant".
78
-
While the term MessageType might imply a specific message format, in this context, it effectively designates the role a message plays in the dialogue.
71
+
For instance, OpenAI recognizes message categories for distinct conversational roles such as "`system,`" "`user,`" or "`assistant.`"
72
+
While the term, `MessageType`, might imply a specific message format, in this context, it effectively designates the role a message plays in the dialogue.
79
73
80
74
For AI models that do not use specific roles, the `UserMessage` implementation acts as a standard category, typically representing user-generated inquiries or instructions.
81
-
To understand the practical application and the relationship between Prompt and Message, especially in the context of these roles or message categories, please refer to the detailed explanations in the Prompts section.
75
+
To understand the practical application and the relationship between `Prompt` and `Message`, especially in the context of these roles or message categories, see the detailed explanations in the <<Prompts>> section.
82
76
83
77
=== AiResponse
84
78
@@ -95,11 +89,11 @@ public class AiResponse {
95
89
96
90
The `AiResponse` class holds the AI Model's output, with each `Generation` instance containing one of potentially multiple outputs from a single prompt.
97
91
98
-
The `AiResponse` class additionally carries a map of key-value pairs providing metadata about the AI Model's response. This feature is still in progress and is not elaborated on in this document.
92
+
The `AiResponse` class also carries a map of key-value pairs providing metadata about the AI Model's response. This feature is still in progress and is not elaborated on in this document.
99
93
100
94
=== Generation
101
95
102
-
Finally, the `Generation` class contains a String representing the output text and a map that provides metadata about this response.
96
+
Finally, the `Generation` class contains a `String` that represents the output text and a map that provides metadata about this response:
103
97
104
98
105
99
```java
@@ -114,30 +108,38 @@ public class Generation {
114
108
115
109
== Available Implementations
116
110
117
-
These are the available implementations of the `AiClient` interface
111
+
The `AiClient` interface has the following available implementations:
118
112
119
-
* OpenAI - Using the https://github.com/TheoKanning/openai-java[Theo Kanning client library].
120
-
* Azure OpenAI - Using https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview[Microsoft's OpenAI client library].
121
-
* Hugging Face - Using the https://huggingface.co/inference-endpoints[Hugging Face Hosted Inference Service]. This gives you access to hundreds of models.
122
-
* https://ollama.ai/[Ollama] - Run large language models, locally.
113
+
* OpenAI: Using the https://github.com/TheoKanning/openai-java[Theo Kanning client library].
114
+
* Azure OpenAI: Using https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview[Microsoft's OpenAI client library].
115
+
* Hugging Face: Using the https://huggingface.co/inference-endpoints[Hugging Face Hosted Inference Service]. This gives you access to hundreds of models.
116
+
* https://ollama.ai/[Ollama]: Run large language models locally.
123
117
124
118
Planned implementations
125
-
* Amazon Bedrock - This can provide access to many AI models.
126
-
* Google Vertex - Providing access to 'Bard', aka Palm2
119
+
* Amazon Bedrock: This can provide access to many AI models.
120
+
* Google Vertex: Providing access to 'Bard' (AKA Palm2).
127
121
128
-
Others are welcome, the list is not at all closed.
122
+
Others are welcome. The list is not at all closed.
129
123
130
124
== OpenAI-Compatible Models
131
125
132
126
A variety of models compatible with the OpenAI API are available, including those that can be operated locally, such as [LocalAI](https://github.com/mudler/LocalAI). The standard configuration for connecting to the OpenAI API is through the `spring.ai.openai.baseUrl` property, which defaults to `https://api.openai.com`.
133
127
134
-
To link the OpenAI client to a compatible model that utilizes the OpenAI API, you should adjust the `spring.ai.openai.baseUrl` property to the corresponding URL of the model you wish to connect to.
128
+
To link the OpenAI client to a compatible model that uses the OpenAI API, you should adjust the `spring.ai.openai.baseUrl` property to the corresponding URL of the model you wish to connect to.
135
129
136
130
== Configuration
137
131
132
+
This section describes how to configure models, including:
133
+
134
+
* <<openai-api,OpenAI>>
135
+
* <<azure-openai-api,Azure OpenAI>>
136
+
* <<hugging-face-api,Hugging Face>>
137
+
* <<ollama-api,Ollama>>
138
+
139
+
[[openan-api]]
138
140
=== OpenAI
139
141
140
-
Add the Spring Boot starter to you project's dependencies
142
+
Add the Spring Boot starter to you project's dependencies:
141
143
142
144
[source, xml]
143
145
----
@@ -148,7 +150,7 @@ Add the Spring Boot starter to you project's dependencies
148
150
</dependency>
149
151
----
150
152
151
-
This will make an instance of the `AiClient` that is backed by the https://github.com/TheoKanning/openai-java[Theo Kanning client library] available for injection in your application classes.
153
+
This makes an instance of the `AiClient` that is backed by the https://github.com/TheoKanning/openai-java[Theo Kanning client library] available for injection in your application classes.
152
154
153
155
The Spring AI project defines a configuration property named `spring.ai.openai.api-key` that you should set to the value of the `API Key` obtained from `openai.com`.
154
156
@@ -159,12 +161,13 @@ Exporting an environment variable is one way to set that configuration property.
159
161
export SPRING_AI_OPENAI_API_KEY=<INSERT KEY HERE>
160
162
----
161
163
164
+
[[azure-openai-api]]
162
165
=== Azure OpenAI
163
166
164
-
This will make an instance of the `AiClient` that is backed by the https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview[Microsoft's OpenAI client library] available for injection in your application classes.
167
+
This makes an instance of the `AiClient` that is backed by https://learn.microsoft.com/en-us/java/api/overview/azure/ai-openai-readme?view=azure-java-preview[Microsoft's OpenAI client library] available for injection in your application classes.
165
168
166
169
The Spring AI project defines a configuration property named `spring.ai.azure.openai.api-key` that you should set to the value of the `API Key` obtained from Azure.
167
-
There is also a configuraiton property named `spring.ai.azure.openai.endpoint` that you should set to the endpoint URL obtained when provisioning your model in Azure.
170
+
There is also a configuration property named `spring.ai.azure.openai.endpoint` that you should set to the endpoint URL obtained when provisioning your model in Azure.
168
171
169
172
Exporting environment variables is one way to set these configuration properties.
There is not yet a Spring Boot Starter for this client implementation, so you should add the dependency to the HuggingFace client implementation to your project's dependencies.
183
+
There is not yet a Spring Boot Starter for this client implementation, so you should add the dependency to the HuggingFace client implementation to your project's dependencies and export an environment variable:
180
184
181
185
[source, xml]
182
186
----
@@ -192,11 +196,12 @@ There is not yet a Spring Boot Starter for this client implementation, so you sh
192
196
export HUGGINGFACE_API_KEY=your_api_key_here
193
197
----
194
198
195
-
Obtain the endpoint URL of the Inference Endpoint. You can find this on the Inference Endpoint's UI https://ui.endpoints.huggingface.co/[here].
199
+
Obtain the endpoint URL of the inference endpoint. You can find this on the Inference Endpoint's UI https://ui.endpoints.huggingface.co/[here].
196
200
201
+
[[ollama-api]]
197
202
=== Ollama
198
203
199
-
There is not yet a Spring Boot Starter for this client implementation, so you should add the dependency to the Ollama client implementation to your project's dependencies.
204
+
There is not yet a Spring Boot Starter for this client implementation, so you should add the dependency to the Ollama client implementation to your project's dependencies:
200
205
201
206
[source, xml]
202
207
----
@@ -209,7 +214,7 @@ There is not yet a Spring Boot Starter for this client implementation, so you sh
209
214
210
215
== Example Usage
211
216
212
-
A simple hello world example is shown below that uses the `AiClient's generate method that takes a `String` as input and returns a `String` as output.
217
+
The following listing shows a simple "Hello, world" example. It uses the `AiClient.generate` method that takes a `String` as input and returns a `String` as output:
0 commit comments