-
Notifications
You must be signed in to change notification settings - Fork 131
Oracledb connector information #289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Oracledb connector information #289
Conversation
@Minal-Agashe-Oracle please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
Learn Build status updates of commit f4d9819:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracledb-connector.md | Details | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md | ✅Succeeded |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracledb-connector.md
- Line 0, Column 0: [Warning: pivot-group-missing - See documentation]
Missing metadata attribute: 'zone_pivot_groups'. To use zone pivots in your file, you must specify the valid pivot group that contains the pivot IDs you want to use.
- Line 1, Column 1: [Warning: author-missing - See documentation]
Missing required attribute: 'author'. Add the current author's GitHub ID.
- Line 1, Column 1: [Warning: title-missing - See documentation]
Missing required attribute: 'title'. Add a title string to show in search engine results.
- Line 1, Column 1: [Warning: ms-author-missing - See documentation]
Missing required attribute: 'ms.author'. Add the current author's Microsoft alias.
- Line 1, Column 1: [Warning: ms-topic-missing - See documentation]
Missing required attribute: 'ms.topic'.
- Line 0, Column 0: [Suggestion: filename-incomplete - See documentation]
The filename contains an incomplete word, misspelled word, acronym, or abbreviation that is disallowed.
- Line 1, Column 1: [Suggestion: ms-date-missing - See documentation]
Missing required attribute: 'ms.date'.
- Line 1, Column 1: [Suggestion: description-missing - See documentation]
Missing required attribute: 'description'.
- Line 1, Column 1: [Suggestion: ms-service-suggested]
Missing required attribute: 'ms.service'. ms.service is suggested for Yaml:Mime type: Module.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
Learn Build status updates of commit e5eb914:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracledb-connector.md | Details | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md | ✅Succeeded |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracledb-connector.md
- Line 0, Column 0: [Warning: pivot-group-missing - See documentation]
Missing metadata attribute: 'zone_pivot_groups'. To use zone pivots in your file, you must specify the valid pivot group that contains the pivot IDs you want to use.
- Line 1, Column 1: [Warning: author-missing - See documentation]
Missing required attribute: 'author'. Add the current author's GitHub ID.
- Line 1, Column 1: [Warning: title-missing - See documentation]
Missing required attribute: 'title'. Add a title string to show in search engine results.
- Line 1, Column 1: [Warning: ms-author-missing - See documentation]
Missing required attribute: 'ms.author'. Add the current author's Microsoft alias.
- Line 1, Column 1: [Warning: ms-topic-missing - See documentation]
Missing required attribute: 'ms.topic'.
- Line 0, Column 0: [Suggestion: filename-incomplete - See documentation]
The filename contains an incomplete word, misspelled word, acronym, or abbreviation that is disallowed.
- Line 1, Column 1: [Suggestion: ms-date-missing - See documentation]
Missing required attribute: 'ms.date'.
- Line 1, Column 1: [Suggestion: description-missing - See documentation]
Missing required attribute: 'description'.
- Line 1, Column 1: [Suggestion: ms-service-suggested]
Missing required attribute: 'ms.service'. ms.service is suggested for Yaml:Mime type: Module.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
Learn Build status updates of commit f1e93b3:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md | Details | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md | Details |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md
- Line 50, Column 1: [Warning: file-not-found - See documentation]
Invalid file link: './oracledb-connector.md'.
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md
- Line 2, Column 1: [Warning: ms-author-missing - See documentation]
Missing required attribute: 'ms.author'. Add the current author's Microsoft alias.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
Learn Build status updates of commit f53fd6a:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md | Details | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md | ✅Succeeded |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md
- Line 2, Column 1: [Warning: ms-author-missing - See documentation]
Missing required attribute: 'ms.author'. Add the current author's Microsoft alias.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
Learn Build status updates of commit 474b117:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md | Details |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md
- Line 2, Column 1: [Warning: ms-author-missing - See documentation]
Missing required attribute: 'ms.author'. Add the current author's Microsoft alias.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
Learn Build status updates of commit f7afe13:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md | Details | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/TOC.yml | ✅Succeeded |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md
- Line 2, Column 1: [Warning: ms-author-missing - See documentation]
Missing required attribute: 'ms.author'. Add the current author's Microsoft alias.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
| Supported vector property types | <ul><li>ReadOnlyMemory\<float\></li><li>Embedding\<float\></li><li>float[]</li><li>ReadOnlyMemory\<double\></li><li>Embedding\<double\></li><li>double[]</li><li>ReadOnlyMemory\<short\></li><li>Embedding\<short\></li><li>short[]</li><li>ReadOnlyMemory\<byte\></li><li>Embedding\<byte\></li><li>byte[]</li><li>BitArray</li><li>BinaryEmbedding</li></ul> | | ||
| Supported index types | <ul><li>Flat (default)</li><li>HNSW</li><li>IVF</li></ul> | | ||
| Supported distance functions | <ul><li>CosineDistance</li><ul><li>FLOAT32, FLOAT64, and INT8 vector default</li></ul><li>CosineSimilarity</li><li>DotProductSimilarity</li><li>NegativeDotProductSimilarity</li><li>EuclideanDistance</li><li>EuclideanSquaredDistance</li><li>HammingDistance</li><ul><li>BINARY vector default</li></ul><li>ManhattanDistance</li><li>JaccardSimilarity</li></ul> | | ||
| Supported filter clauses | <ul><li>==</li><li>!=</li><li><</li><li><=</li><li>></li><li>>=</li><li>List.Contains() <ul><li>Only when checking for the model property is in the list</li></ul></li></ul> | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| Supported filter clauses | <ul><li>==</li><li>!=</li><li><</li><li><=</li><li>></li><li>>=</li><li>List.Contains() <ul><li>Only when checking for the model property is in the list</li></ul></li></ul> | | |
| Supported filter clauses | <ul><li>==</li><li>!=</li><li><</li><li><=</li><li>></li><li>>=</li><li>List.Contains() <ul><li>Only when checking if the model property is in the list</li></ul></li></ul> | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Minal-Agashe-Oracle Let's accept this change.
@@ -0,0 +1,277 @@ | |||
--- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also check out the index.md file in the same folder as this file. It has a table of all the connectors, which can be updated for .net.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! We're working on updating the index.md as well.
| Restricted at least one vector | No | | ||
| Allow multiple primary keys | No | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Restricted at least one vector
is already covered by Supports multiple vectors in a record
.
And since none of the connectors support multiple primary keys today, we can probably leave that out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Minal-Agashe-Oracle Let's consolidate these two items into a single line item. That should make the points clear.
Supports zero, one, or multiple vectors in a record = Yes
| Restricted at least one vector | No | | ||
| Allow multiple primary keys | No | | ||
|
||
Vector data searches require Oracle Database 23ai. All other Oracle connector features are available using Oracle Database 19c or higher. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider putting this in an important highlight
Vector data searches require Oracle Database 23ai. All other Oracle connector features are available using Oracle Database 19c or higher. | |
> [!IMPORTANT] | |
> Vector data searches require Oracle Database 23ai. All other Oracle connector features are available using Oracle Database 19c or higher. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Minal-Agashe-Oracle We can accept this change.
Add the Oracle Database Vector Store connector NuGet package to your project. | ||
|
||
```dotnetcli | ||
dotnet add package Oracle.VectorData --prerelease |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the package released yet? Was going to test the instructions out, but I don't see the package on nuget yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@westey-m Not yet. It's likely to be released within the next week.
```csharp | ||
using Oracle.VectorData; | ||
|
||
var collection = new OracleCollection<string, Hotel>("<connection string>"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't this one also require a collection name param?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch!
@Minal-Agashe-Oracle You can replace the line of code with
var collection = new OracleCollection<string, Hotel>("<connection string>", "skhotels");
|
||
The Oracle Database Vector Store connector provides a default mapper when mapping data from the data model to storage. This mapper does a direct conversion of the data model properties list to the Oracle database columns to convert to the storage schema. | ||
|
||
The Oracle Database Vector Store connector allows the application to provide a custom mapper in combination with a `VectorStoreCollectionDefinition`. In this case, the `VectorStoreCollectionDefinition` can differ from the supplied data model. The `VectorStoreCollectionDefinition` is used to define the database schema, while the data model is used to interact with the vector store. A custom mapper is required in this case to map from the data model to the custom database schema defined by the `VectorStoreCollectionDefinition`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does it still provide a custom mapper? FYI: We did drop this for the other connectors, since it doesn't really work with the linq based filtering.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@westey-m No, we don't provide a custom mapper. VectorData provides basic custom mappings that use VectorStoreCollectionDefinition.
For LINQ-based filtering, Oracle uses table metadata and data model to filter columns. We took care of this in our metadata layer (mappings between data model to Oracle type) in the Oracle connector.
Our implementation may be different than other connectors. What scenarios do not work well with LINQ-based filtering? We can check those situations out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are not supporting custom mappers, no need to worry. It was something we had originally, but we dropped it. It doesn't work well with linq based filtering, since the model that the user provides and that the linq query is over, does not match the internal db schema, when using a custom mapper.
My comment was really around the fact that the docs here are referring to a custom mapper, and should be revised if there is none.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@westey-m
Is storage schema definition using record definitions the long-term plan to provide data mapping?
@Minal-Agashe-Oracle
Replace the current paragraph:
The Oracle Database Vector Store connector allows the application to provide a custom mapper in combination with a VectorStoreCollectionDefinition. In this case, the VectorStoreCollectionDefinition can differ from the supplied data model. The VectorStoreCollectionDefinition is used to define the database schema, while the data model is used to interact with the vector store. A custom mapper is required in this case to map from the data model to the custom database schema defined by the VectorStoreCollectionDefinition.
with the following paragraph instead:
The Oracle Database Vector Store connector supports data model annotations and record definitions. With annotations, developers provide information to the data model for creating indexes and database column mapping. With record definitions, the information can be defined and supplied separately from the data model.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is storage schema definition using record definitions the long-term plan to provide data mapping?
@alexkeh, the type of mapping that we had previously supported via custom mappers was e.g. turning nested structures into flat ones and similar complex remapping. Record definitions in my mind is more about defining the schema, and there is typically a fairly standard 1 to 1 mapping from schema to model that we follow.
Learn Build status updates of commit f625c1d: ✅ Validation status: passed
For more details, please refer to the build report. |
Learn Build status updates of commit b236df2: ✅ Validation status: passed
For more details, please refer to the build report. |
Learn Build status updates of commit e61bb76: ✅ Validation status: passed
For more details, please refer to the build report. |
Learn Build status updates of commit 0bd30e5:
|
File | Status | Preview URL | Details |
---|---|---|---|
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md | Details | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md | ✅Succeeded | ||
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/TOC.yml | ✅Succeeded |
semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/oracle-connector.md
- Line 191, Column 224: [Warning: hard-coded-locale - See documentation]
Link 'https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/schema-with-record-definition?pivots=programming-language-csharp' contains locale code 'en-us'. For localizability, remove 'en-us' from links to most Microsoft sites.
- Line 191, Column 224: [Suggestion: docs-link-absolute - See documentation]
Absolute link 'https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/schema-with-record-definition?pivots=programming-language-csharp' will be broken in isolated environments. Replace with a relative link.
For more details, please refer to the build report.
Note: Your PR may contain errors or warnings or suggestions unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
…efinition in the oracle-connector md file.
Learn Build status updates of commit 2a0ba77: ✅ Validation status: passed
For more details, please refer to the build report. |
@microsoft-github-policy-service agree company="Oracle" |
Added new file oracledb-connector.md
This is the file path:
/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors
Updated index.md (overview) file.
This is the file path:
/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors